Memory changes over time - each time we remember something, we rewrite it, so it actually changes! It is like transcribing from old film into newer, higher quality film, but can come with transcribing errors. A bit like all these camera phones with different filters each time! So often there are events that happen in life that siblings remember very differently - indeed even as to who the event actually happened too. Each can claim it was themselves when in fact it only happened to one of them.
Use those photos to re-embed the memories. As philosophers say, everything is a dream. The past is a dream, the future is a dream. Even the present is a dream, because our senses only pick a very small amount of the world we are actually part of, so we don’t know what reality actually is, just how we have interpreted it in that moment.
As we get older I think we get more used to apologising for our faulty memories. I personally think our brains are rather like filing cabinets, when it comes to memories. The longer we have lived the more full they are, so there is less space to store new memories, and old, scarcely used memories get very cobwebby, are harder to find, and, like paper, have yellowed, got tattered and are harder to read.
As for your dad. Try talking to people who would have known him, and combine that with the photos you have. Rebuild the picture of him. It might not be completely accurate, but in many ways that doesn’t matter, as long as it is true to his essential character. It is also quite interesting to hear how others perceived people - as it builds a broader picture of who that person was.
My guess, with the books, it wasn’t an important memory, so was just very shallowly stored - why fill up your mind with the inessential? Memory of your dad is clearly important to you, so work on rebuilding that.
And, I think it is important to keep some photographs of our past, particularly as we get older. It is so easy, in say middle age, to do a major declutter and get rid of many. I’m at that stage. In later life they help to recall those earlier years, and remember our journey to who we are now.