Yes. Also the priming pump piston acts on the same pumping chamber and uses the same two valves, if the seal on the priming pump piston is leaking, and you can see that it has in the pics, then air can get in there also. I think they have also seal when the pump handle is screwed all the way down, so make sure that cap is screwed down as tight as you can with your fingers.
I have seen those hand pumps mentioned do that also. The pressure relief valve I mentioned is in the fitting just in front of and below the number 1 injector line. It has a steel and a rubber hose connected to it. I have had a good bit of problems with algae in older machines. An inline automotive filter protects the pump but requires frequent changes. Killem will take care of the algae but then you need to buy filters be the case. The supply pump does not need a filter that fine anyway, just a screen or bird catcher. The big spin on, you are replacing anyway, filters the fuel for the engine. My local fuel injection company sells a clear inline filter with a screen in it. The screen can be removed and cleaned. That is how I would start.
1. New fuel filter and inline screen.
2. Check and or replace check valves, #3. If you sand them that's like 2000 grit paper on a piece of glass.
3. Inspect pressure regulator, I thank it has one.
4. If problem persists replace hand primer pump.