7. AI-Assisted Coding

🎯 Learning Goals

• Explain the purpose of prompt engineering and its use cases

• Describe the limitations of large language models like ChatGPT

• Design prompts with specificity and technical language to improve responses

📗 Technical Vocabulary

• Prompt Engineering

• Artificial Intelligence

• Large Language Models

• Tokenization

• Hallucinations

What is Prompt Engineering?

Introduction

• AI is changing the way we program. Tools like ChatGPT can help with debugging, code generation, and understanding complex programming concepts.

• Large Language Models (LLMs) like ChatGPT, Google Gemini, and Claude are powerful coding assistants when used effectively.

• In this lesson, we’ll explore how to craft high-quality prompts to get accurate and useful responses. Why? Because the way you talk to these tools makes all the difference in how useful they are! We’ll share tips and tricks to make sure your AI game is on point.
  

Demystifying LLMs

• LLMs are essentially very fancy autocomplete systems. They rely on patterns, the context you provide, and huge amounts of training data to predict the next word in a sentence.

• LLMs break down text into smaller pieces called tokens, which the model uses to understand and generate language. Tokens can be as small as individual characters, but are sometimes whole words! For example, the sentence “I love coding!” might be tokenized like this:

• LLMs like ChatGPT process and analyze text at the token level, using patterns in these tokens to predict the next value in a sequence. For example, if you write “The best name for a dog is“ the LLM predicts the most likely tokens to come next, based on its model of human language. In the image below, you can see that while the model selected “Fido” to complete the sentence, there were other likely tokens that could come next such as “R”, “B”, “Spot”, or “Max.” It’s important to note that LLMs are generally set to include some randomness, so it doesn’t always select the most likely token! This is why when you ask an LLM the same question twice, you will likely get slightly different responses.

• Once the LLM selects the next token, it doesn’t stop there! Now, it uses the entire original sentence plus the new token to predict the next token after that and so on. This leads to a butterfly effect where small differences in the starting state of a system lead to very different outcomes. In the example below you can see that since the sentence was completed with “Fido,” the remaining tokens follow that idea.

• However, if the model had completed the sentence with “Spot” instead, the rest of the response would have been different as well.

• These examples begin to illuminate some of the limitations of AI tools like Large Language Models. The AI is simply guessing the next word based on statistical patterns in its training data, but it can’t really “think” like humans do.

💭

Think About It

Now that you know about how LLMs work, why do you think ChatGPT generated this incorrect response?

From the AI's perspective, "strawberry" is not a sequence of individual letters, but a sequence of token IDs.

Miscounting the 'R's in "strawberry" shows how humans and AI see text differently. We read text naturally, letter by letter, but AI tools, like ChatGPT, chunk text into tokens that combine multiple letters or even entire word parts.

Knowing this helps you use AI better! It’s a reminder that while these models are super smart, they don’t "think" like we do. That’s why they sometimes mess up even on easy stuff.

Other Limitations of AI

• Beware of biases! These models are trained on large datasets that reflect societal biases.

• Watch out for hallucinations. LLMs sometimes make stuff up! Because of the tokenization process, LLMs sometimes generate text that seems realistic, but is actually inaccurate or misleading.
• Think of it like a friend who’s super confident about random facts but sometimes just makes stuff up when they don’t know the answer. For example, if you asked, "Who invented pizza?" and the AI said, "Pizza was invented by aliens in 1850"—that’s a hallucination. It’s not lying on purpose; it just doesn’t know the answer and guesses based on patterns in its training data.
• These hallucinations happen because AI doesn’t "know" things the way humans do. It doesn’t have facts stored like a library—it predicts what sounds right based on the data it’s been trained on and the previous context. Sometimes those predictions are spot on, but other times... not so much!

• Limited knowledge. LLMs are essentially a snapshot of the world’s knowledge at the moment of their training. For GPT-4 Turbo, the training data cut-off was December 2023. This means the model typically does not have knowledge of recent events unless specifically enabled with a "browse the internet" feature.

• AI tools are fantastic at generating ideas and speeding up workflows, but they’re not magic. They’re super advanced word machines that still rely on your input. That’s why learning to craft effective prompts is such a big deal—it helps you get the most accurate and useful results from AI.

Privacy Considerations

Writing Effective Prompts

General Techniques

• ✅ Be Specific — Specify the problem in detail

• ✅ Use Technical Language — Include relevant SQL terms and concepts

• ✅ Provide Context — Describe the goal and constraints

Example: SQL Query Generation

Iterative Prompting

1. Prompt

2. Evaluate

3. Revise/Follow-Up

Potential Pitfalls

• You aren’t getting the result you want after several revisions.

• You no longer understand the code.

• You are caught in a Copy, Paste, and Cross Your Fingers loop.

‼️ The “Copy, Paste, and Cross Your Fingers” Loop

Controlling Response Format

• Summarize the purpose of SQL joins in 3 bullet points.

• List the steps to process CSV files.

• Create a table comparing SQL and Tableau.

• Explain how Tableau and SQL work together using an analogy.

• Describe hallucinations in the context of large language models in 30 words or less.

• Give me an acronym to help me remember the structure of a case statement in SQL.

Other Tips

1. Don’t overthink it! Just get started. You can always redirect after getting an initial response. That’s the great thing about using a language model compared to other tools — it’s a conversation!

2. If your question is complex, break it into smaller steps! The AI will be more accurate if you break up complex tasks into more manageable steps.

3. Ask for explanations. If AI generates code, request a line-by-line breakdown as comments within the code block.

4. Verify AI-generated code. AI solutions are not always correct. Read the output carefully and test it before using.

💼 Takeaways

• Prompt engineering is the art of crafting precise, clear, and specific instructions for AI tools

• LLMs use tokenization for processing text and predicting the next word in the sequence

• Use specificity, technical language, and context to improve responses

Next Steps

For a summary of this lesson, check out the 7. AI-Assisted Coding One-Pager!