How long does it take to learn the prerequisites to learn these if you don't already have a CS background?
Reading through the list, I can barely parse half the titles due to acronyms and terms I've never seen before. Would I be able to figure them out along the way? Or is there a body of knowledge this is assuming I should already have?
(This isn't a criticism of the list, or even the idea of presenting such a list without explaining who the target audience is and isn't. I'm genuinely wondering, if I just start at the top, is there any hope I would figure this out, or do I need to start somewhere else entirely?)
Depends on what you mean by learn... And what other relevant background one has outside of CS - programming/math/stats/systems/engineering. Assuming one has a few years of that, and one want to get to the level of being able to apply this to a problem that comes up in a realistic setting (making a product etc). Maybe extreme dedication over many months, or a few years at 20 hours a week? This is assuming that one either is very good at studying alone, or have access to people one can ask alone the way.
If one has no relevant background, then I would 5+ years of full time study.
I think I may just need to accept that the work it would take to get to this level would mean a whole life change.
It's been 20+ years since I took calculus in high school, and I haven't taken a math or science class since. I reckon that puts me at least five years behind where I'd need to be to start this course and the listed ones.
Thanks for the recommendation, and helping me assess whether it's the use I want to make of my time!
The papers have bibliographies with citations in text. The knowledge you need to read a given paper usually lies within 3 degrees of bibliographical separation.
You can also use reverse citation search on Google scholar to find newer works that cite a given work. This can uncover a review or textbook, which often makes the concept much simpler to digest. And it can be a more advanced approach which gives you an alternative view or lets you abandon the original paper altogether.
2. Back to tha basics
For math, it would help if you have college level calculus, differential equations, linear algebra, and statistics. If you don't have that, be ready to assign yourself math readings and homework. It would also really help if you had analysis, topology, abstract algebra, and category theory under the belt, just so you can see the mathematical forest for the trees... But that's not really so necessary
You can often find the author's preferred reference for a given work in the bibliography, and use that.
For computer science, it helps to be able to read algorithm pseudocodes, follow complexity and correctness proofs, and understand performance experiments and charts. Skimming Knuth's "Art of Computer Programming" should be very helpful.
You'll need some algorithmic/statistical learning theory as well, since authors often use these concepts to prove that a system can learn. Though TBH, I usually skip these proofs and focus on the performance details. I usually find a higher performance paper before I need to care about the learning proofs. Still it helps to expect them and have an idea of how they work, if only to know when you can safely ignore them.
3. Narrative Arcs
Many AI results build on keystone papers that define eras. We can build on these papers by understanding their philosophy and deeper implications, even if we don't really understand their math. There's a story that you will tell yourself, and it may lead to interesting research or business results.
For example, if you read "Attention is all you need", you set the stage for LLMs. then "LLMs are in context learners", followed by the "chain of thought", "reflexion", and "toolformer" paper will give you a great sense of how LLMs are emerging as more than just stochastic parrots, ready to be developed and deployed in a lot of ways that overcome their issues. But looking at how bad the numbers can be even with our best efforts, you also get the sense of how much work these systems will need to actually be good, and the sense that corner cutting will lead to a lot of pretty awful systems.
It's a fun time to be a computer scientist or software engineer! Lots of cool papers to read!
Wow! I'm glad there are people like you out there to be excited about it! I glazed over before I even got to the end of your comment, with the dazzled sort of awestruck glaze where I realize I will never do even a tiny fraction of the work required to know things about this topic you probably take totally for granted.
Thanks for taking the time to expose me to your enthusiasm!
Reading through the list, I can barely parse half the titles due to acronyms and terms I've never seen before. Would I be able to figure them out along the way? Or is there a body of knowledge this is assuming I should already have?
(This isn't a criticism of the list, or even the idea of presenting such a list without explaining who the target audience is and isn't. I'm genuinely wondering, if I just start at the top, is there any hope I would figure this out, or do I need to start somewhere else entirely?)