My mind goes round and round and round to the same ideas and the same connections but I am finding it very hard to out them all together. There is something very deep and fundamental that we are missing in biology, the reason why theories like relativity do not exist.
First is the stupidity of biologists trying to use statistical methods to "average out" historical events to eliminate "accidents". This all stems from Popper and his view of science, which is alright for physics but not appropriate for Biology. You ignore history in a biological problem at your peril. This was never clearer to me than at an ISMB meeting where I had been subjected to the same talk twice, by two different post-docs from the same group (the P.I. was on the organising committee) where the presenter was asked about how well the method worked on real data. The reply was that it didn't. It only worked on the model data. At that point I gave up on ISMB meetings because they are too incestuous.
The problem is that that synthetic data is all nNormals and Poissons and nothing like reality. Reality has nasty things like extinctions, bottle-necks, frozen accidents etc. Reality has population structures, isolation, incest, dispersion and above all else a richness of history that you cannot average out. This is why Mayr was so sceptical about the "bean counters" like Haldane, Fisher and Sewall-Wright. So trying to impersonate physics in quantitative biology throws the baby out with the bath-water. This is not to deny statistics its role but biology is full of what Taleb calls Black Swan problems. These are the out of the usual events, the unknown unknowns that mean the unexpected has a disproportionally large effect. So our knowledge and possibility of knowledge in biology is limited by the barrier of experience that Hume identified.
The error of ignoring history is compounded by excess reductionism that fails to look at the system. How can I judge the fitness of an organism in isolation from its environment? That is why in vitro and in vivo experiments differ so much (see Holmes on Viral Evolution). There have been those who have worked in solving these problems but usually they have been marginalised by the reductionist, statistical main-stream. Waddington stands out amongst these with his epigenetic landscape for development, but more important than that is extending this to evolutionary landscapes through evolutionary canalisation. Monod made some contributions towards this in Chance and Necessity where he introduced gratuity as a side-effect when there is a change in the landscape so evolution has access to new areas. Kaufman put these ideas into a computational and numerical frame-work. The recent appearance and very rapid disappearance of systems biology also produced a burst of activity particularly in the landscape view from Kitano.
The basis for where we need to go is already written but nobody has put it together in a useful way. There are so many works that are ignored and connections missed. These problems are hard - they frustrated Darwin.
Waddington - Towards a Theoretical Biology 4 volumes (The bible of real systems biology).
Waddington - The strategy of the genes (Canalisation - if an evolutionary biology text does not cite him do not believe them).
Taleb - Anti-fragile (His Magnum Opus about Black Swan problems and solving them).
Haldane - On the Causes of Evolution (One of the three founding works of evolutionary genetics).
Maynard-Smith - Evolution and the Theory of Games (Shows that mixed solutions work - not the message most take from it).
Axelrod - The Evolution of Cooperation (Tit-for-tat is the best and cooperation gives benefits).
Sewall-Wright - Evolution and the Genetics of Populations 4 volumes (His complete work on population genetics).
Kimura - The neutral theory of molecular evolution (An important assumption of coalescence).
Holmes - The Evolution and Emergence of RNA viruses (Quite specific and not always right but some good points).
Kaufman - Origins of Order (Has quite a lot of holes but it is a start).