2025-03-04 Week 8 - Session 2 - TDS Jan 25#

Duration: 0h 47m

Here’s an FAQ summary of the Tools in Data Science (TDS) live tutorial session:

Q1: What is the purpose of this live tutorial session?

A1: This session is primarily for addressing your queries, facilitating discussions, and gathering suggestions regarding the TDS program and projects. It’s not a knowledge-sharing session for this week but a dedicated space for question resolution and open dialogue.

Q2: For Project 2, what’s the recommended approach for solving the graded assignment questions: using multiple function calls or an LLM-based auto-coding solution?

A2: While using 50 separate function calls for 50 questions is a possible but challenging approach, we are looking for you to demonstrate how you can deploy an LLM agent to solve the problems. The best approach would be an LLM-based auto-coding solution where the LLM directly provides the answers. We are exploring different options and encourage creative solutions, but the LLM-based approach is preferred.

Q3: Can you clarify the LLM-based approach for Project 2? Does it involve prompt engineering based on error feedback?

A3: Yes, that’s correct. The LLM-based approach involves feeding any errors or issues back to the LLM and using prompt engineering to refine its responses and arrive at the correct answers.

Q4: Regarding Project 2, if we use the IIT-provided API key, what happens if the $2 credit limit is reached during testing or evaluation, leading to failures?

A4: That’s a valid concern regarding tokenization and credit limits. Currently, it’s set at $2 per month. We will check with the team and provide a definitive update on the discourse forum within a day regarding the API call limits and how this will be handled during testing and evaluation.

Q5: In Project 2, if the POST request includes the API key during testing, would that resolve the credit limit issue? My token would be used, and if $2 is crossed, evaluation could fail.

A5: We understand your point. This shouldn’t be a challenge, but we will confirm the specifics regarding API calls and credit usage during evaluation on the discourse forum within a day.

Q6: When will the Project 1 solution code be showcased, as mentioned by Anand Sir?

A6: The Project 1 solution showcase should happen within a week or so. We are currently coordinating with Anand Sir to confirm his availability and schedule. We will notify you well in advance via email and the discourse forum.

Q7: My computer crashed during the ROE (Review of Essential Skills) submission due to too many open tabs after working for 24 hours. I couldn’t submit my answers. What happens in such a situation?

A7: That’s an unfortunate operational challenge. We strongly advise everyone to submit their answers at regular intervals, even if you’ve only completed one or two questions. This creates intermediate submissions, and we keep a track of all successful submissions. Your highest score from all submissions is considered. We had a GMeet call to address technical issues during the ROE, but we couldn’t intervene in individual system crashes. For future reference, always try to restart your system and keep it free of unnecessary processes before critical tasks to avoid such issues.

Q8: In ROE Question 1, I successfully submitted an answer via an HTTP link. But after closing the tab and returning, it showed as incorrect. Why did this happen, and what can I do?

A8: As long as you successfully submitted and saved your answer, even if the tab was closed, we have a record of it. We track all individual student submissions. If you believe there’s a discrepancy, please document it in the discrepancy form. We will review your submissions and have a cross-discussion if needed. The key is always to submit your answers to ensure they are recorded.

Q9: For Project 2, will the evaluated answers need to be exact matches to the graded assignments (GAs), or can different files be used during evaluation? Some GA questions involve specific files.

A9: The GAs are randomized, meaning not everyone gets the exact same parameters or files. Your solution needs to be flexible enough to handle changes in parameters and files. The files themselves will be fixed in their format (e.g., ABC.csv), but their content or specific instances can change. The logic of your solution should be robust enough to process these parameterized inputs correctly.

Q10: Can the LLM be used to determine file names and integrate them into function arguments in Project 2?

A10: Yes, ideally. Any “objects” like file names can be parameterized. Your solution’s core logic should be accurate, and these parameters (file names, data) can then be passed to that logic. The LLM could assist in identifying or handling these dynamic inputs.

Q11: What type of image format and handling should we expect for Project 2 questions involving image manipulation? Will they be attachments in POST requests, or Base64 encoded within the question itself?

A11: Most images in our segments are embedded as URLs or JPG objects. Typically, images are handled as binary file attachments in POST requests. However, if a question provides a Base64 encoded string of an image directly within the question, your solution should be able to process that too. We will clarify the exact expected method (attachment vs. in-question Base64) for specific image manipulation questions on the discourse forum.

Q12: In Project 2, some GA questions involve GitHub repository creation, actions, and deployment (e.g., using Vercel). How should we handle these, as they seem to require manual intervention or might conflict with Vercel limitations?

A12: You can automate GitHub operations using GitHub CLI (Command-Line Interface) commands. Git commands allow you to create repositories, manage actions, and perform most operations that you would typically do through a GUI. While this can involve user authentication (e.g., passing tokens as environment variables), it avoids manual intervention. We will provide detailed guidance on using GitHub CLI for these tasks.

Regarding Vercel, you raised a valid point about potential limitations for heavy processes like LLM execution, Ngrok, or extensive GitHub CLI actions. Please raise these specific concerns on the discourse forum. We will clarify the expected approach and how our evaluation script handles such scenarios programmatically.

Q13: For GA1, Question 6 (using Dev Tools), how will a question with hidden elements (using JavaScript) be passed in a plain-text POST request for evaluation?

A13: This is a good question. I need a bit more time to look into the specifics of handling JavaScript-hidden elements via plain-text POST requests for evaluation. I will investigate the CLI options for Chrome Dev Tools and provide a detailed response on the discourse forum.

Q14: Some GA questions require using external software like Open Refine or specifically state that tasks (e.g., Google Sheets, Excel) must be done only in those tools. How do we incorporate this into our LLM-based solution, or handle cases where other methods give different answers?

A14: For tasks like data cleaning, entity resolution (similar to Open Refine), or any operations typically done in Excel/Google Sheets, Python provides ample libraries (e.g., spaCy, NLTK for entity resolution; Pandas for data manipulation) that can perform these programmatically. There is no operation feasible in Excel that cannot be done in Python.

However, you also highlighted that some questions explicitly state that they must be solved only in Google Sheets or Excel, implying specific outputs. This requires careful consideration for programmatic solutions. Please raise these specific tool-dependent question requirements on the discourse forum. We will clarify how these scenarios will be handled by the evaluation script programmatically and what exact approach is expected from your side.

Q15: When evaluating Project 2, will the parameters and files for GA questions be fixed, or will they be randomized? For example, if a SQL question specifies certain parameters or a file, will it remain the same for everyone?

A15: The GA questions are randomized. This means that not everyone will receive the exact same set of parameters or files. Your solution must be flexible and parameterized to correctly handle varying inputs. The core logic of your solution should be able to process different files and parameters (e.g., different file names for common row identification) provided by the evaluator.

Remember to submit all your project-related questions on the discourse forum for official and detailed responses.