How AI Really Works: A Federal Employee's Guide to Large Language Models 

“Any sufficiently advanced technology is indistinguishable from magic.”  

— Science fiction writer and futurist, Arthur C. Clarke 

AI isn’t magic, but it’s close enough to feel that way. Whether you’re an AI wizard already performing daily feats, an eager adopter still learning the ropes, or someone avoiding it entirely, one thing is certain: understanding how AI works makes it less intimidating and more powerful. 

This blog post pulls back the curtain on Large Language Models (LLMs), the engines powering tools like ChatGPT, Claude, Gemini, and Copilot. The more you understand how these systems work, the better you can harness their benefits while protecting against their risks. 

What Are Large Language Models? 

Large Language Models (LLMs) are AI systems trained on vast amounts of text so they can recognize patterns in language and generate human-like responses. At their core, they predict what word—or sequence of words—is most likely to come next, based on everything they’ve learned from books, articles, code, and conversation. 

When we talk about tools like ChatGPT, Claude, Gemini, Grok, Perplexity, or Copilot, we’re really talking about proprietary LLMs developed by OpenAI, Anthropic, Google (Alphabet), xAI, Perplexity AI, Inc., and Microsoft. While they began as advanced text-prediction and translation systems, they’ve evolved into flexible reasoning engines capable of summarizing, analyzing, explaining, and generating new content. 

That evolution was driven as much by hardware as by software. A major inflection point came when NVIDIA introduced specialized AI infrastructure—starting with the DGX-1 in 2016, followed by the H100 chip in 2022 and the Blackwell platform in 2024. These systems made it possible to train and run models at an unprecedented scale and speed. 

The result can feel almost supernatural. But it isn’t. What’s happening under the hood is mechanical, statistical, and—once you break it down—entirely understandable. 

The Mechanics: Tokens, Predictions, and Patterns 

Think of LLMs as hyper-advanced autocomplete systems. They predict the next word, or, more precisely, the next token, based on patterns learned from massive amounts of training data. 

What’s a token? It’s the smallest unit of text the model processes, usually a word or part of a word. For example, ‘unbreakable’ might split into three tokens: ‘un,’ ‘break,’ and ‘able.’ On average, one word equals about 1.3 to 1.5 tokens. 

When you write a prompt, the LLM breaks your input into tokens, assigns each a numerical ID, and uses those IDs to predict what should come next. This process repeats, token by token, until the response is complete. 

If it feels like you are interacting with a human, that is by design, but the model doesn’t ‘understand’ language the way humans do. Instead, it uses statistical relationships between words to make educated guesses. During training on trillions of text examples, the model constantly adjusts its internal settings—called parameters—to improve future predictions. Over time, these adjustments teach it grammar, facts, and even basic reasoning. 

The Training Process: Turning Chaos into Intelligence 

Imagine a giant wall of knobs and dials, all starting at random settings. As the model trains, each incorrect prediction tweaks the dials slightly, gradually dialing in settings that produce better results. 

The model’s architecture, called a transformer, uses a mechanism called self-attention to focus on the most relevant parts of your input, even if they’re far apart in the text. This is how it maintains context and generates coherent, human-like responses. 

LLMs can write emails, translate languages, summarize documents, generate code, and more, without being reprogrammed for each task. They don’t ‘know’ the answer in advance; they generate responses step by step, predicting the best next token at each stage. 

The Hidden Cost: Power, Water, and Environmental Impact 

NVIDIA chips excel at massive parallel computations, allowing LLMs to process inputs and generate outputs at astonishing speeds. But this comes at a cost. 

Data centers housing these AI systems consume tremendous amounts of electricity and cooling water. For federal agencies, these resource demands are not just environmental considerations—they translate directly into budgetary pressure, infrastructure planning, procurement constraints, and regulatory exposure.  

Your Next Steps: From Understanding to Action 

Now that you know how LLMs work, the “magic” turns out to be mathematics, enormous amounts of data, and powerful hardware working together. The more you understand AI, the less intimidating it becomes, and the more effectively you can use it. 

Ready to explore what AI can do for you and your agency? 

• Try it yourself: Ask an LLM to help with a routine task at work or home 

• Learn more: Explore Management Concepts’ AI training programs designed for federal employees 

• Get expert guidance: Contact Management Concepts for consultation on integrating AI into your operations 

AI is a tool for you to master and use. As you explore this emerging environment with all its exciting opportunities, let Management Concepts be your source for training and consultation. 

For more essential guidance on how Artificial Intelligence is redefining the federal landscape, rely on Management Concepts. Start your journey here.   

Dave B. Doane is a scholar-practitioner of the Value-Focused Thinking methodology in Decision Analysis. Excel Power Platform with the @RISK plug-in and a Python compiler are his tools in trade. After serving a career in the US Army, Mr. Doane founded a small business and continues to serve primarily Federal clients as a consultant, instructor, author, and speaker.  

For Management Concepts, Dave is a subject matter expert and instructor in the domains of Analytics, Program and Project Management, and Finance. Dave and his wife Beth make their home in Fairfax, Virginia. In their free time, they love motorcycling on the beautiful byways and boating in the nearby Bay.