You don’t need to be an engineer to make good decisions about AI, but you do need the vocabulary. Vendors pitch “agentic” add-ons, staff ask whether they can paste a client letter into a chatbot, and Microsoft asks whether you want Copilot on every license.
If you learn only five
- Large language model: the engine behind ChatGPT, Claude, Gemini and Copilot. It predicts text, which is why it can sound sure and be wrong.
- Hallucination: confident, false output. Anything that matters needs a human check.
- Grounding: connecting AI to your own files. Answers get better, and the AI can also see every file the user has access to.
- Agent: AI that takes actions, such as sending email or editing files, as well as answering questions. The more it can do, the more its permissions and human review count.
- Training on your data: some personal plans, such as ChatGPT Free and Plus, use what you type to improve the AI unless you turn that off. Check the plan before anyone types in client data.
Basic AI terms
| Term | What it means |
|---|---|
| Artificial intelligence (AI) | Software that performs tasks we associate with human thinking, like understanding language, recognizing images or making predictions. The label covers everything from spam filters to chatbots. |
| Machine learning | The most common way AI is built today. The software learns patterns from large amounts of example data instead of following hand-written rules. |
| Generative AI | AI that creates new content, such as text, images, audio, video or code, based on patterns in the data it was trained on. |
| Large language model (LLM) | The engine behind most text-based AI tools. Trained on huge amounts of text, it generates responses by predicting what words should come next. GPT, Claude and Gemini are families of LLMs. |
| Chatbot or AI assistant | The app you type or talk to. ChatGPT, Claude, Gemini and Copilot are assistants built on top of LLMs. |
How you talk to AI
These four terms explain why the same tool gives great answers one day and poor ones the next.
| Term | What it means |
|---|---|
| Prompt | The instructions or question you give the AI. Clear prompts with context, examples and the format you want get much better results. |
| Token | The chunks of text an AI reads and writes, roughly a word or part of a word. Many AI services price and limit usage by tokens. |
| Context window | How much text the AI can consider at once, including your prompt, attached files and the conversation so far. In a very long conversation, earlier details can drop out. |
| Multimodal | An AI that works with more than text, such as reading a photo of a receipt, listening to a meeting recording or describing a chart. |
How AI gets its answers, and gets them wrong
| Term | What it means |
|---|---|
| Training data | The material a model learned from. Its built-in knowledge stops at a cutoff date, so it may not know recent events, laws or prices unless it can search. |
| Grounding | Connecting an AI to specific sources, like your SharePoint files, a website or a database, so its answers draw on that material. |
| Retrieval-augmented generation (RAG) | The common technique for grounding. |
| Fine-tuning | Further training a model on specialized examples so it handles a particular task or style better. Most small organizations won’t need it. |
| Hallucination | When an AI confidently states something false, like a made-up statistic, court case or citation. NIST’s generative AI risk profile calls this “confabulation.” It is a built-in tendency of how these models work, so expect it to stay. |
The practical rule: treat AI output as a first draft from a fast, well-read assistant who sometimes makes things up. Someone who knows the subject should check anything headed to a client, a regulator or a court.
AI that takes action
| Term | What it means |
|---|---|
| Copilot | Microsoft’s brand for AI assistants built into its products. Microsoft 365 Copilot works inside Word, Excel, Outlook and Teams and is grounded in your organization’s data. |
| AI agent or agentic AI | AI that carries out multi-step tasks on its own, such as reading an inbox, updating a spreadsheet and drafting replies. |
| Model Context Protocol (MCP) | An open-source standard for connecting AI applications to outside systems like files, databases and business apps. Its own documentation compares it to a USB-C port for AI. It is what lets an assistant reach into your other tools. |
| Human in the loop | A design where a person reviews or approves what the AI does before it takes effect. For money, client communication or records, make this the default. |
The more an AI can do, the more its permissions matter. An agent that can send email or edit files can make mistakes, or be tricked, at machine speed. Our own help desk follows the human-in-the-loop rule: AI classifies and triages every new ticket, and a technician reviews each one before work starts.
Security and privacy terms
| Term | What it means |
|---|---|
| Shadow AI | AI tools staff use without approval, often free personal accounts. It is the most common way client data ends up somewhere it shouldn’t. |
| Training on your data | Whether a provider uses what you type to improve its models. OpenAI, for example, uses ChatGPT Free and Plus conversations for training unless you turn that setting off, and says it does not train on Business or Enterprise workspace content by default. |
| Data boundary | Where your prompts and files are processed and stored. Microsoft says Microsoft 365 Copilot keeps prompts, retrieved data and responses within the Microsoft 365 service boundary and doesn’t use them to train its foundation models. |
| Permissions oversharing | Copilot and similar tools show users anything they already have access to. Loose file permissions become easier to discover. |
| Prompt injection | Hidden instructions planted in a web page, email or document that trick an AI into doing something it shouldn’t, such as leaking data. OWASP ranks it the top risk for LLM applications. |
| Deepfake | AI-generated audio, video or images that imitate a real person. The FBI has warned that criminals increasingly use AI-generated voices to impersonate people their targets know and trust. |
| AI acceptable use policy | Your written rules for which AI tools staff may use, what data can go into them, and who reviews the output. |
Shadow AI and acceptable use policies are where we spend most of our AI time with clients: writing the policy, then blocking or controlling the tools it rules out. Our post on shadow AI goes deeper, our guide to AI governance covers the policy itself, and how to spot deepfakes covers the voice and video side.
Questions to ask before you adopt an AI tool
- Does the provider train on our data, and can we turn that off on our plan?
- Where is our data processed and stored, and for how long?
- Will they sign the agreements we need, such as a business associate agreement for health data?
- What can the tool access and do inside our systems, and can we limit it?
- Who reviews the output before it reaches a client?
For healthcare practices, law firms, accounting firms and nonprofits holding donor records, those answers decide whether a tool is usable for client work at all. Once you know which tasks you want AI for, our guide to AI for everyday tasks is the next read.
When a vendor’s pitch uses three of these terms in one sentence, you are welcome to bring it to us. Book a 20-minute call and we will translate it, or read more about our AI services.
Sources: NIST AI 600-1, AI RMF Generative AI Profile, July 2024; Microsoft Learn, Data, privacy, and security for Microsoft 365 Copilot; OWASP GenAI Security Project, LLM01:2025 Prompt Injection; OpenAI Help Center, Data Controls FAQ; Model Context Protocol, What is MCP?; FBI IC3 PSA, Senior US Officials Impersonated in Malicious Messaging Campaign, May 2025.
