Here’s an FAQ-style transcription of your live tutorial session:

Q1: Can we change the Git repo after the deadline?

A1: No, that won’t be possible. The deadline has already been extended to the end of today. If you could modify the repository after the deadline, it would essentially mean the deadline is ignored. So, any changes you want to make should be done now.

Q2: Is there a chance to increase the AI pipe limit as I’ve reached it?

A2: We discussed this this morning, and we didn’t find anyone who has actually breached the AI pipe limit yet. However, we will increase the limit if needed during evaluation. Gemini, for example, has daily and hourly limits that you can check in the pricing section.

Tip: If you are doing local testing, use Ollama for a shortcut. After testing, you can deploy your application with the AI pipe token or OpenAI API keys. We keep a log of any responses indicating that limits have been reached, and if we encounter users with this issue, we will investigate. We will reissue or raise the limit if necessary before evaluation.

Q3: When converting JSON to MB, it says “maximum retries exceeded” for both conversion and embeddings. Now, when testing the image with cURL, it shows the same prompt.

A3: You are likely hitting the daily limits of the Gemini API provider you are using. If you exhaust your daily limit, it will respond with that error. For local testing, use Ollama. When ready to deploy, use an AI pipe token or OpenAI keys.

Q4: My YAML file is working with OpenAI Gemini and API key. NPX commands also work. Does this mean I’m good to go, or do I need to make other changes?

A4: Your YAML file working means you’ve configured it correctly. It doesn’t necessarily mean your project submission is working fine. To check your project submission, you need to test that YAML file against your project, or use the cURL command against your project.

The YAML file we provided has a slight error, which you’ll need to fix on your own. If your YAML file and cURL commands are working, then your project is good to go.

Q5: The test endpoint provided in the mail gives a “wrong” answer for the GS score (expected 110, my code gives a different answer). If I test it with cURL or by downloading the YAML file locally, my answer is correct. It seems NPX prompt foo is using a cached answer. When you test on your end, will “no-cache” be run with the NPX command?

A5: Yes, if you use --no-cache at the end of the command, it will correct the answer. It’s possible that the web server part of the hosted Prompt Foo solution is also caching, so there might be a double cache issue. We are aware of the --no-cache option. If your local code gives the right answer with “no-cache” and the website doesn’t, it means your code is fine, and the expected answer is correct.

Warning: Don’t try to overfit your code to the current YAML file. We will be using robust private test cases for evaluation, not the exact same YAML file. So, even if the provided YAML file has minor issues, these will be addressed in the new YAML used for evaluation. If your local setup is running and the answer from your code matches how the YAML file expects it, then your code is correct. For some people, there might be a CORS issue. If you’re getting responses everywhere (e.g., from another laptop), then CORS is likely not an issue.

Q6: I received the email with the endpoint, but I can’t open it anywhere. It doesn’t have a UI. How are we supposed to test using it? I’m using cURL commands to ask random questions, and the endpoint is answering with multiple links. So it’s working in practice. How do I test using this endpoint?

A6: There might have been a mistake in the email where a colon was used instead of a dot in the endpoint URL. It should be a dot. The instructions on how to use it are provided within the email itself.

Q7: Can the AI Proxy key be reset? For some reason, my application says I have enough quota, but when I check AI Proxy usage, it’s very high (because I did a lot of retries for embeddings, resulting in 11MB embedding size but a lot of tokens used).

A7: Before evaluation, we will raise the limits for whichever token (AI Proxy or AI Pipe) is being used. We also keep a log of any responses that indicate limits have been reached, and if encountered, we will look into it to re-issue or raise limits.

Q8: How do I test the endpoint that Jivraj was talking about?

A8: You need to put your API endpoint in the provided site and then enter your AI Proxy key. The YAML testing only involves about four test cases, so it won’t use many tokens (around 1000 tokens).

Q9: When running my YAML file, I’m getting an error: “Your authentication token is not from a valid issuer.”

A9: This means your YAML file is not configured correctly. You need to specify whether you are using AI Pipe Proxy, ChatGPT, Claude, etc. There is documentation on the Prompt Foo website on how to configure it for custom AI providers. The token we provide is free to you, and it’s a custom provider we’ve set up, not directly from OpenAI. If the key were directly from OpenAI, no extra configuration would be needed. Since it’s a custom provider, you must configure the YAML file correctly to use the key and the right URL for the provider.

Q10: While submitting the project, what if the GitHub repo I’m submitting is different from the Git on which my Vercel deployment is running? Will this be a problem? Do I need to submit the same Git that Vercel is using?

A10: If your Vercel deployment is a subset of your original Git repo (meaning the Git repo is a superset of your Vercel deployment), then it’s fine.

Q11: The cURL request returns correctly for me, but the evaluation script (Prompt Foo) that is given doesn’t show 110 for the test case. It shows it for cURL, but not for Prompt Foo. Should I not be worried?

A11: Don’t worry too much about the Prompt Foo result. As long as some cases are passing and you’re getting some valid responses, your configuration is okay. The Prompt Foo file is not the ultimate evaluation. It was just provided as a tool to help you check if your application is responding as expected. Even if you get 100 out of 100 on that file, it won’t make a difference as we will run our own private test cases. If the cURL request is giving the correct output, it will be fine. Don’t worry about Prompt Foo not giving full marks; partial marking will be applied.

Q12: I ran the cURL request locally, and it’s giving “no relevant answer could be found” or “no relevant data found for this question” for generic questions about TDS. I also used Prompt Foo, and it’s giving “no files found for variable image” in the first question.

A12: This sounds like a logic error in your RAG application design. You might have an issue in how you’ve created your RAG. I suggest you watch the project sessions again, or at least the section on RAG and embeddings, and ensure you’ve recreated the project correctly.

Q13: While submitting the URL on the portal, it says “failed to fetch,” but Prompt Foo and cURL are working.

A13: This is likely a CORS (Cross-Origin Resource Sharing) issue on your application. You need to add a CORS middleware to your service. If you are hosting it on Hugging Face, you also need to add a proxy middleware. If everything else is working but you get “failed to fetch,” it often points to a CORS issue or an incorrect HTTP method (e.g., using GET instead of POST for an endpoint).

Q14: If my code is giving 75% accuracy, is that fine?

A14: Yes, 75% accuracy is fine. The point is that we know your application is responding to the Prompt Foo file, it’s being fetched across the internet, and it’s producing an output that Prompt Foo expects.

Q15: Does the Prompt Foo file (evaluation page) have the right questions? One of the questions I’m seeing is “When is the end term for September 2023?”

A15: Don’t worry about this public Prompt Foo file. It was not heavily validated and was just provided as an aid to check if your application is running as expected. We will be using a robustly constructed Prompt Foo file with carefully selected test cases for actual evaluation, checking things like time range, assertions, etc. We will also perform in-house testing before evaluating you. If your cURL command is working fine, just relax.

Q16: In TDS website, if I do the check-all option, sometimes it’s correct within 13 seconds, sometimes 50 seconds, sometimes beyond that. Will it cause any problems in my project? What should be the ideal acceptance time for the check?

A16: The ideal acceptance time is two minutes. If your deployment launches within two minutes, it’s fine. Just make sure when you hit the submit button, it passes both checks. If you already scored two marks (meaning it passes both checks), don’t resubmit anything, we will take the latest submission.

Q17: Will we be marked on our GitHub repo also, or only on the API endpoint?

A17: There are two aspects. We will test your API endpoint with Prompt Foo. Your GitHub repo should contain the code that allows us to build your application. There are some bonus marks for those sorts of things. Your GitHub repo should contain the same code as your deployed application.

Q18: You said there’s a logical error in my application itself. Does “application” mean the entire directory, or app.py, or the specific file I’m using?

A18: When I say “logical error,” it has to do with how your RAG retrieval is being done. Your description of the problem suggests it’s not retrieving an expected response. This could be because cosine similarity isn’t working correctly, or your embeddings haven’t worked correctly. Somewhere along those lines, something has gone wrong. You’ll need to check those sections of your code carefully.

Q19: Do we have to upload all scraped JSON files?

A19: No, I don’t think so, because you would have anyway created some markdown and embeddings files. So uploading scraped JSON files wouldn’t be necessary.

Q20: The RAG component is still not working for me, and test cases are not passing. I’ve watched all the sessions. Can you tell me from where I can learn more, and if I can fix it in an hour or two?

A20: You can use the videos on the Tools in Data Science website that specifically talk about RAG and embeddings. Just go over the technical details there. RAG can be a bit more difficult. It’s not about prerequisites, but you need to take the time to understand how it works. If you try to do this project at the last minute, it will be very difficult. You should spend at least a couple of days to understand how these various components fit together. If you didn’t spend time understanding it over the past three weeks, you’ll find it challenging now. You could ask ChatGPT to give you a high-level explanation of RAG, then elaborate on specific bullet points, or even speak to it if you have a paid subscription. This can help you quickly understand the issue.

The key is to correctly create the chunks; that’s the first hurdle. If chunks aren’t created properly, RAG won’t be effective. Then, ensure embeddings are created without issues, and finally, retrieve embeddings using cosine similarity. Creating embeddings themselves takes a lot of time. Even with years of experience, you’ll find this section very challenging if you’re new to AI. Even if you understand it, creating it within an hour is difficult.

Even if you can’t submit Project 1 successfully, don’t give up. Project 2 will also contain LLM-based encoding. The concept of RAG is very new, and many people are experiencing this new workflow for the first time. It takes time to understand. The skills you take from this (understanding RAG) will be much more valuable than the grade you get. If you want to submit something and are short on time, you could try implementing just the ChatGPT part (without RAG embeddings) or a simplified version of embeddings. It’s challenging to do this by yourself. Collaborate with others.

Q21: Is there a specific time limit for evaluation?

A21: We haven’t set a specific time limit yet, but it will be soon after the deadline. We will send out an email when the evaluation window starts, asking you to keep your service running between specific times.

Q22: My AI pipe quota is exhausted. My code will work properly once it’s renewed. Can I still submit it if it says “failed to fetch” due to the quota being exhausted?

A22: “Failed to fetch” will be an issue if it prevents you from getting the two marks for submission. However, AI pipe errors specifically due to quota exhaustion are okay as we will track those. Make sure you get the two submission marks. Even if you’ve added CORS middleware and it still says “failed to fetch” (but cURL works), you can still submit if it successfully gets the two submission marks on the portal.

Q23: My GitHub repo is different from my Vercel deployment because Vercel was too heavy. Is this okay?

A23: Yes, that’s fine, as long as your GitHub repo is a superset of your Vercel deployment.

Q24: After I submit the link and the deadline passes, if I make changes to my GitHub repo, will that affect my submission?

A24: Yes, we will be able to see any changes you make because we have your GitHub repo. It records every single change you make.

Q25: The grading document says projects will be evaluated on an 80/20 basis (80% by instructors, 20% by peer reviews). Is this correct?

A25: No, that’s not accurate. It’s a mistake in the grading document. The projects will be completely programmatically evaluated. If there’s a subjective component, it will be evaluated by an LLM grader, not a human.