What Will be Left After the AI Bubble Pops?

Every bubble pops eventually. When this one goes, what will be left behind?

Well, there will be little models – Hugging Face, Llama, etc – that run on commodity hardware. The people who are learning to “prompt engineer” these “toy models” have gotten far more out of them than even their makers imagined possible. They will continue to eke out new marginal gains from these little models, possibly enough to satisfy most of those low-stakes, low-dollar ap­plications. But these little models were spun out of big models, and without stupid bubble money and/or a viable business case, those big models won’t survive the bubble and be available to make more capable little models.

There are some promising avenues, like “feder­ated learning,” that hypothetically combine a lot of commodity consumer hardware to replicate some of the features of those big, capital-intensive models from the bubble’s beneficiaries. It may be that – as with the interregnum after the dotcom bust – AI practitioners will use their all-expenses-paid education in PyTorch and TensorFlow (AI’s answer to Perl and Python) to push the limits on federated learning and small-scale AI models to new places, driven by playfulness, scientific curiosity, and a desire to solve real problems.

There will also be a lot more people who un­derstand statistical analysis at scale and how to wrangle large amounts of data. There will be a lot of people who know PyTorch and TensorFlow, too – both of these are “open source” projects, but are effectively controlled by Meta and Google, respectively. Perhaps they’ll be wrestled away from their corporate owners, forked and made more broadly applicable, after those corporate behemoths move on from their money-losing Big AI bets.