'had __ed' doesn't just say 'hey, this was earlier'.
(honestly, it would be completely frivolous to have a whole separate tense for this purpose, since context will almost always make the order of events amply clear. also, it would create serious problems if the order of the events wasn't known or wasn't relevant.)
'had __ed' generally serves 2 purposes:
1/
to emphasize the fact that something had been completed, or had created some sort of effect, on the situation of the past-tense action:
When the fire started, all of the employees had already left the building. (the impact of the employees' departure here is pretty clear.)
2/
to describe something ongoing up to the timeframe of the past-tense action, and directly affected by that action:
After 25 years, Roy finally found the inner peace that he had been seeking. (he had been seeking it ... right up until the moment when he found it.)
clearly, these things are hard to describe, and you should NOT attempt to learn them as 'rules'. instead, just read a large number of examples of them in well-written sources, and pay close attention to context.