What is MAoSS?
MAoSS is a neural network that is finetuned to score responses from the Short Story task in several languages. MAoSS matches human ratings more closely than DSI.
MAoSS currently supports the following languages: Arabic, Chinese (Mandarin), Dutch, English, French,Farsi, German, Hebrew, Italian, Polish, Russian, and Spanish.
How do I use MAoSS?
To have your short story responses scored, upload a utf-8 encoded CSV that has a column named 'item' (e.g., stamp-letter-send) and another column named 'response', which has the participant's response to the short story item. The algorithm is case sensitive for the item/response column names, but it does not matter if other columns are present in the input CSV.
The algorithm will add two new columns to your CSV: one named 'prediction' which reflects the model's creativity score and a column named 'modelname', which is added to reflect the model providing the score.
If the language you wish to score in is not currently supported, we recommend translating the items/responses to English via Google Translate. We've found MAoSS performs quite well in that scenario.
