JB

I’d like to talk a bit more about AI. But to get there, I’ve got to wind up the pitch a bit.

One of the most important lessons I’ve learned is to ask “who profits from this?”, especially when evaluating fantastic claims or considering extraordinary events. Taking claims at face value and assuming beneficial intent, especially from the rich and powerful, is foolish. There must be verifiable facts to back up the claims, otherwise they are trying to sell you something. That “something” might be a good or service, or it might be that they are trying to sell you on an idea or a concept. Sometimes it’s both. Either way, grand announcements to the world aren’t made without an agenda, and it’s important to understand the aims and goals of that agenda.

We are living in interesting times, whether we like it or not. Interesting in that we are hearing extravagant, unverifiable claims regularly, extraordinary in that we are actually seeing events happening that affect our lives. Noteworthy, because the claims and the facts of what we can see with our own eyes and experience are not lining up. There are many examples of this, but I said I wanted to talk about AI so let’s get on with it.

Some Terminology

Words have meaning, and it’s important to use the correct words when attempting to convey an idea, especially one involving highly technical concepts. You might consider this pedantic, to which I would say yes.

So, Artificial Intelligence is a field of study involving building computer systems capable of performing tasks that would normally require human intelligence. I think this is a good and worthy pursuit, and that we should be spending time trying to make our machines smarter and better at the tasks we want them to do.

Machine Learning, according to Wikipedia:

“is a field of study in artificial intelligence concerned with the development and study of statistical algorithms that can learn from data and generalize to unseen data, and thus perform tasks without being explicitly programmed.” Great work has been done here in many applications from agriculture to medicine to credit card fraud detection.

Large Language Models, the “LLM” that most of us talk about when talking about AI, are a specific focus of Deep Learning, under the umbrella of Machine Learning. LLMs take massive amounts of data to train, and the training and hosting of the LLMs is driving the data center boom across the US.

When training an LLM, the model is shown a piece of text, and asked to predict what comes next. When it gets the answer wrong, an algorithm adjusts the numerical weights in the model to improve the next prediction. This process is repeated trillions of times, using as much training data as they can. Like, the entirety of the internet, every book, every published paper, everything the developers of the technology can get their hands on.¹ The training establishes a mathematical understanding of words, phrases, concepts, and patterns of logical reasoning.

When asked a question, the LLM breaks the question into small chunks called tokens. It then processes those tokens using the mathematical representation of an established body of understanding it learned during training, and then constructs, bit by bit, the most likely next word in its response. The question, additional input data like documents, and any online searches the LLM might do during the chat session are known as context, and all of it is processed through the same mathematical model. These LLMs are incredibly advanced, and incredibly complex, so these two paragraphs are a vast oversimplification of what’s actually going on, but I believe it’s broadly correct.

An “AI agent” is a piece of software that communicates with an LLM. It has no AI in itself, it merely communicates a local state with the LLM and executes commands based on the response of the LLM. An LLM has no agency, and cannot determine on its own what it wants to accomplish. It uses its mathematical model to determine next steps in accomplishing a goal given to it in a prompt, and can use an agent to execute commands in pursuit of that goal.

What we see in Claude and ChatGPT is the application of LLM technology to a product. The companies behind these two applications, Anthropic and OpenAI, along with, to a lesser extent, Google’s Gemini, SpaceXAI’s Grok, and a few others, are making the LLM technology available to the general public, and we don’t really know what to think of them yet.

Motivation

I’m fascinated by the technology of LLMs, but at the same time I take great issue with how the LLMs are being developed and deployed by these companies. From SpaceXAI building methane gas turbines in Memphis to OpenAI’s alarming lack of control over their technology, most visible in the hack of Hugging Face. They are burning cash (and the environment) at an alarming rate, and, in my opinion, can’t be trusted.

In September, the Financial Times reported:

“OpenAI expects its expenses will far outrun its revenues, which are projected to grow tenfold in the same period, from $36bn this year to $350bn in 2030. In total the company anticipates booking $840bn in revenue between now and the end of 2030.”

Those are big numbers to be lighting on fire, especially with no expected profit in the next five years. This means that they need to keep investment cash coming in to keep the company going, so they promise the moon and make astounding claims, and then build new data centers to try to deliver on those claims.

At the same time they’re trying to convince the general public, and more importantly, investors, that AI will fundamentally change our lives, and that everyone needs AI embedded deeply in their company. I’ve seen many organizations drink this Kool-Aid, some are even letting people go because of the promise, not yet the reality, of AI.

The conversation around AI has an almost cult-like religious phrasing to describe the future the technology promises. A future without work, where all disease has been cured, climate change is solved, and world hunger is eliminated, but at the same time it’s powerful enough that it might just cause the apocalypse.

That’s obviously not the future any of us want, or what I believe is the most likely 10 to 20-year scenario. But, when considering the motivations of the AI companies, the claims start to make sense.

I don’t believe that these companies are going to build an Artificial General Intelligence (AGI), but I do believe that they need as many people in the world as possible to think that they can so they can keep that influx of cash coming in. When they claim that AI can destroy the world, or that AI is too dangerous, what I hear is “aren’t we impressive?”

Think it Through

I’ve gone back and forth over several months exploring the capabilities of AI, and have become familiar with its quirks and shortcomings. AI excels when it can be guided, challenged, and corrected. AI does poorly when it is given ambiguous prompts in a field where the person asking has little expertise of their own to fall back on. It also does better when there’s a large, public corpus to draw from, so it’s great with Python programming, but less good with building a CloudFormation template for a newer AWS service. The more obscure your niche, the more chance that AI will have holes in its knowledge.

Since AI does best when you already know what to do, it does have the ability to supercharge a senior engineer. However, it can be disastrous in the hands of a junior engineer lacking the skills and experience to be able to guide the AI in architecturally sane directions. If you’re just learning the language, AI can be helpful if prompted in a way that will help you learn, but if you just give it a job to do with an agent and walk away, you learn nothing and the AI can write spaghetti code all day. What the AI writes may be equivalent to what the junior engineer would have written, but in the process they gain no experience to help them actually become senior engineers.

The results so far have been that companies are hiring far fewer junior and entry-level engineers than before. That’s not a great trend, since how are we supposed to get more senior engineers without the junior engineers to take their place? The system is flawed.

Additionally, reliance on AI has been shown to have a negative effect on cognitive ability in general. According to Duke:

While AI enables rapid data analysis at an unprecedented speed and scale, overreliance on AI can erode an individual’s critical thinking skills. In the higher education context, researchers have found that university students who use Large Language Models (LLMs) to complete writing and research tasks experienced reduced cognitive load but demonstrated poorer reasoning and argumentation skills compared to those students using traditional search methods.

If we look farther down the chain and into K-12 schools, the scenario is far worse. Kids have very little motivation to put the effort into thinking things through at all when they know they can use their school-issued laptop to ask ChatGPT to do whatever the assignment is for them. According to the Institute of Education Sciences:

General-purpose AI use may hinder students’ learning, particularly if it reduces students’ cognitive effort or replaces student thinking.

Putting in the effort is the point.² And, maybe that’s the entire point. Our human ability to learn and grow hinges on us doing things that are hard. Programming is hard. Reading dry, boring technical documentation is hard. Learning new skills is hard. But, the payoff is worth it.

But again, when considering a big companies push into education, ask “who profits from this?” It’s clearly not the students, who give up their opportunities to learn at the very time when they need to be learning. It’s also not the schools, who need to hire an IT person to administer the laptops, wireless network, and infrastructure for the school. But, the companies profit from making students and schools dependent on their technology. “Hook ‘em when they’re young, and they’ll be yours for life” seems to be the prevailing attitude.

Be Careful Out There

Some people in some situations can benefit greatly from AI, and I wouldn’t be surprised to see that those folks are the technology’s biggest cheerleaders. But not everyone, and not in every situation. The massive corporations can’t be trusted, the technology itself needs an experienced person to prompt it and double-check it, and using the technology irresponsibly has a detrimental effect on our cognitive abilities.

AI was built by stealing everything off the internet, and can be used to mimic the artists it stole from. No attribution is available because it was all thrown into this massive math soup. The leadership of these companies has had no problem cozying up to (wannabe) authoritarians, and seems to have no regard for the effect of their technology or their data centers on the general public. No one asked for this, and now that it’s here we’re not sure what to do with it.

My answer, so far anyway, is to use the technology sparingly, and be critical of whatever it gives you. It might be right, it might not. It is getting better, but I wonder if we are reaching the peak of what’s possible with this fork of LLM AI development. If so, what we have is useful sometimes, annoying at others, and outright dangerous when used indiscriminately.

If you decide to use AI or not is up to you. My advice is simply that if you do use it, be careful with it. Don’t allow a robot to think for you.