Rubrik (100 points) for Final Report

We are not evaluating if you succeeded or failed at accomplishing what you intially decided to do. It is totally fine if your results are negative and not significant. The important criteria we evaluate your project is whether you put a reasonable effort, deeply understand nature and challenges of your problem, set a reasonable hypothesis to tackle the challenges, and are able to clearly communicate them with others. Here is our rubrik for project evaluation:

Below are three general evaluation criteria:

• (10 points) Novelty: Compared to the state-of-the-art methods/systems/datasets, how novel is your approach? Is your work publishable?
• (10 points) Significance: How storng is your result? Is your finding still holding if different setups or prompting tricks?
• (10 points) Clarity: How clear and easy-to-follow is your report? Do you have well organized presetnation of your results and problem definition?

Introduction / Background / Motivation:

• (5 points) What problem do you try to solve? Describe your objectives cleraly without using any technical jargon.
• (5 points) How is it done today by other researchers? What are the limitations and challenges of current practice?
• (5 points) Who might be interested in your work? What kinds of impact can you make?

Approach:

• (5 points) What did you do exactly? How did you solve the problem? Why did you think it would be successful? What is your hypothesis?
• (5 points) What challenges did you anticipate and/or encounter during the development of your approach? Did the very first thing you tried work?
• (5 points) What is scientific novel of your approach to address the challenges?

Experiments / Results / Error Analysis:

• (5 points) How did you measure success? What research questions do you want to validate? What evaluation metrics and experiments were used? What were the results, both quantitative and qualitative? Did you succeed? Did you fail?
• (5 points) No matter you succeed or fail, why? Which data points are incorrectly predicted by yours but previous models can't, or vice versa.
• (5 points) Are there still some failure cases? Why can't your approach address them? Any potential solutions?
• (5 points) Are the ideas/probelm/results presented with appropriate illustration?

Additional points:

• (5 points) Replicability: How easily are your results able to be reproduced by others?
• (5 points) Datasets: Did your dataset or annotation affect other people's choice of research or development projects to undertake?
• (5 points) Ethics: Does your work have potential harm or risk to our society? What kinds? If so, how can you address them?
• (5 points) Discussion: What limitations does your model have? How can you extend your work for future research?

At least some of these questions and others should be relevant to your project and should be addressed in the final report. You do not need to address all of them in full detail. Some may be irrelevant to your project and others may be standard and thus require only a brief mention.