Here’s an FAQ-style summary of the tutorial:

Q1: Is this session being recorded?

A1: Yes, it is being recorded.

Q2: Can you explain what you’re doing? I’m not able to follow. Are you sending the request to the LLM?

A2: I’m setting up a FastAPI server. I’ve added a post endpoint that takes a file as input to upload it to my application. Later, I’ll process it using an LLM. For testing purposes, I’m using Gemini to avoid consuming too many tokens.

Q3: Is there no need to perform data cleaning on the input file, for example, from a Wikipedia source?

A3: No, we don’t have to write hardcoded data cleaning code. The LLM’s job is to write the code for processing, not to do the processing itself. This means it can generate code for tasks like data extraction, manipulation, or querying, which will then run locally on your server. This approach saves on token usage and allows for more complex data handling.

Q4: For better understanding, could you draw a process flow diagram to visualize what’s happening?

A4: Yes, I can illustrate the workflow (diagram was explained at 14:00-16:00 and 21:54-23:00):

  1. User Input: You send a questions.txt file to the API endpoint.
  2. API Processing (LLM Call 1):
    • The API takes the question and sends it to the LLM (GPT) along with a “preamble” (an instruction telling the GPT what to do with the question).
    • The preamble asks the GPT to break down the task into smaller, easily executable chunks, keeping in mind the expected data structure.
  3. LLM Response (Tasks): The GPT returns a list of these executable tasks.
  4. API Processing (LLM Call 2):
    • The API takes these tasks and sends them back to the LLM, asking it to generate Python code for each task.
    • This step might involve a loop to run the generated code locally, handle errors, and make corrections as needed.
  5. LLM Response (Code/Output): The LLM generates the necessary code to process the data.
  6. API Processing (Execution): The API executes the generated code locally on your server.
  7. Final Output: The API sends the processed data as a JSON array back to you (the initiating server/client).

This approach ensures the LLM acts as an “agent” to understand the problem and write solutions, while the heavy processing is done locally, making it more efficient and cost-effective.

Q5: What tools are we using for this project?

A5: We’ll be using Visual Studio Code and FastAPI for the API development. For the LLM, we’ll use Gemini for testing to manage token usage.

Q6: What if the JSON output generated by the LLM is not satisfactory or correct? Will it be validated again by the LLM?

A6: Yes, a validation layer for the LLM’s output can be incorporated. The JSON output doesn’t necessarily go back to the same LLM for re-validation in a direct loop, but you can certainly add steps to ensure quality control.

Q7: Can you clarify the arrows in the diagram? Specifically, the one from API to JSON. What communication does it represent?

A7: The arrows represent the flow of information.

  • The arrow from API to JSON indicates that the API is sending the final processed JSON output back to the client that made the initial request.
  • The output from the LLM (code or data) is sent back to your API, and then your API formats and sends the final JSON response to the client.

Q8: What is the learning outcome of this project? What other learning outcomes can I expect beyond better prompting?

A8: Beyond improving your prompting skills, this project will teach you how to:

  • Implement a robust workflow where the LLM acts as an “agent” to break down complex tasks and generate executable code.
  • Understand that the LLM’s role is to write the code, not execute the heavy data processing itself.
  • Utilize local server resources for data processing (e.g., scraping, data manipulation) after the LLM generates the relevant code.
  • Structure your prompts to elicit specific outputs and actions from the LLM, including generating code and formatting responses.
  • Manage LLM quotas and costs by offloading heavy computational tasks to your local environment.

Q9: Where can I get the Gemini API key?

A9: You can get your Gemini API key from Google AI Studio. Go to their website, and you’ll find a button to “Get API Key” or “Create API Key.”

Q10: My voice might be breaking. Is it better now?

A10: Yes, it is better now.

Q11: The code generated by the LLM (Claude) seems to have messed up the header of the data table it extracted. It also included some extra characters. What should I do about that?

A11: You’re right