Business Writing
Assessment
READ THE INSTRUCTIONS IN THESE SLIDES CAREFULLY. TAKE YOUR TIME.
YOU WILL ONLY GET 1 ATTEMPT ONCE YOU BEGIN THE ASSESSMENT
Overview
We are seeking talented individuals capable of training Large Language Models (LLMs) to solve
real-world problems for the world’s biggest companies.
This writing assessment assesses your skills in:
- Analyzing LLM outputs critically and constructively
- Assessing responses for relevance, accuracy, and safety
- Refining AI systems through detailed, context-aware feedback
- English writing proficiency
Guidelines
Test Integrity Guidelines
To ensure a fair evaluation for all candidates, please adhere to the following:
● No additional monitors allowed
● Use only the provided test environment and IDE
● Close unrelated browser tabs
● No online resources or tools (e.g., LeetCode, ChatGPT, search engines) are permitted
● Consult official language/API documentation only
● Complete the assessment independently, without external assistance
● Respect the integrity of the auto-proctoring system
🚩 Failure to comply with these guidelines WILL result in immediate disqualification. We appreciate your
honesty and commitment to a fair process.
Business Writing Assessment Objective
- Evaluate English writing proficiency:
- Produce clear, effective, and grammatically correct texts
- Craft high-quality rationales
- Analyze model responses comprehensively:
- Articulate advantages and disadvantages
- Identify objective differences
- Maintain appropriate technical terminology
- Demonstrate attention to detail:
- Compare model responses accurately
- Notice and rank subtle differences
Writing Assessment Test Instructions (30 minutes)
- You will be provided a short conversation based on interactions between a user and an AI model
assistant.
- At the end of the conversation, there will be two potential responses given by the AI model to
answer the user’s last message/request.
- Based on the conversation, you are required to
- Read the full interaction between the User and AI model assistant to understand the context.
- Evaluate and compare the two potential responses by the AI model assistant, and write
analyses considering:
- Response evaluation: Here, we need to select the dropdown options and submit our
evaluation for both model responses.
- Preference explanation: Please provide a brief explanation of your preference,
highlighting why you chose one model response over the other.
- Ideal response: Please generate your own ideal response using the 'Code Execution'
cell, as demonstrated in the upcoming slides.
Example
Model Response A: Model Response B:
User: Which option should the
company pursue based on
expected returns? Use a
cost-benefit analysis and calculate
the expected monetary value (EMV)
for each option to determine the
best course of action.
User Response:
Which one would you choose as the best possible model response? Explain why.
Model Response 2 is the best possible answer to the question because it thoroughly explains the cost-benefit analysis, clearly separates
each option's costs and potential returns, and calculates the Expected Monetary Value (EMV) in a logical sequence. Its structured approach
ensures that the reader can easily follow the decision-making process, making it more aligned with the question intent. By providing a
detailed comparison of the outcomes, it delivers a more comprehensive and accurate response to the problem presented.
Correctness: Consider the factuality, logical flow, contextual relevance, and consistency.
Model Response 2 is more informative because it clearly separates the steps in the decision-making process, explains the costs and
expected returns before calculating the EMV for each option, and provides a balanced comparison between the alternatives. It highlights
both the positive EMVs and explains why the Small-scale project is preferable, making it easier for the reader to follow and understand the
decision-making process. In contrast, Model Response 1 is less detailed, making it less informative for someone unfamiliar with decision
tree analysis.
Informativeness: Is the information provided relevant to the user's request? Is the information complete?
Model Response 2 can be considered more correct as per the question asked because it provides a clearer breakdown of both the costs
and returns before calculating the Expected Monetary Value (EMV) for each option. The response follows a structured approach, ensuring
that the decision-making process is transparent and easy to follow. By explicitly stating the cost of each option upfront and giving a more
balanced comparison of the outcomes, it aligns more closely with the intent of the question, which seeks a comprehensive cost-benefit
analysis. Therefore, Model Response 2 more accurately addresses the depth of analysis required by the question.
You’re good to start!
Go back to the assessment tab to begin.
