TransGrading©
A tool for studying how to effectively grade translations and compositions
Copyright © 2011 Sanjun Sun, Ruiliang Guo
All rights reserved.
TransGrading, developed by Sanjun Sun and Ruiliang Guo in 2011, can be used in grading translations and compositions. It allows the user to use holistic and/or analytical scoring methods and to do some calculations. The error category list and holistic category list are customizable. The graded passages can constitute a corpus amenable to further analysis. It is hoped that this tool will help increase the number of empirical studies in different languages or language pairs in the fields of translation quality assessment and writing assessment. Currently, TransGrading is free and portable.
1. About Grading Methods
Basically, there are two kinds of grading methods: holistic and analytical.
The holistic method is also called the general impression method. Graders are usually required to provide for one passage an overall grade or a grade for each holistic category. For instance, in grading a translation, the grader can give a single score from 0 to 10 based on the overall quality of the translation, or give a score (0-10) for translation accuracy and a score (0-10) for expression.
The analytic method works like this: the grader underlines every error in the passage, provides information near the error as to the nature of the error, and deducts points for each error found.
The holistic method is grading on the curve, and involves ranking students in relation to the average performance level. The analytic method is criterion-based; student performance is compared to well-defined criteria, rather than to the performance of other students.
Researchers usually compare the two methods in terms of validity, reliability, and practicality.
2. Product Features
Use holistic and analytical grading methods
Customize the error category list and holistic category list
Calculate error deduction points and total scores
Export errors, error deduction points, and total scores to Microsoft Excel for further analysis and graphical display
Keep track of the time spent on grading
TransGrading runs under the following operating systems:
Windows 7 (32 bit version)
Windows Vista (32 bit version)
Windows XP
4. Starting the Software Program
TransGrading is portable and requires no installation.
To start the program, double click Register.bat and then double click TransGrading.exe. (If you encounter an error when trying to start the program, right click Register.bat and select “Run as Administrator”, and then right click TransGrading.exe and select “Run as Administrator”.)
5. Preparing Your files
TransGrading works with Rich Text Format (RTF) files. The source text, reference text and target text(s) should be in the RTF format. This means that TransGrading can handle any language that your operating system supports. (To launch WordPad in Windows, which natively supports the Rich Text Format, go to Start->Run, type in Write, and press Enter.)
To facilitate data analysis, the target text file must conform to the following pattern:
1) A serial number for each passage precedes the passage. The format is SxxRxxTxxx. S stands for source text, R for reviewer, and T for target text. For example, S31R02T005 indicates that the source text number is 31, the reviewer number is 02, and the target text number is 005. Leave an empty line between the serial number and the passage.
2) If you intend to use the holistic method, put the words Holistic Grade: at the end of each target passage. Otherwise, do nothing.
3) If you intend to keep track of the time spent on grading each passage, put the word Duration: at the end of each passage. Otherwise, do nothing.
There is no alignment requirement. The following are two example target texts.
Click 
to load the source text, which will appear in the top left pane. (In the case of writing research, you can load writing instructions or something else.) Loading the source text is optional.
Click 
to load the reference text (e.g., a reference translation or a model essay), which will appear in the bottom left pane. Loading the reference text is optional.
Click 
to load the target text(s), which will appear in the right pane.
If you have loaded the wrong file or want to load another file, simply click the corresponding icon again and load another file.
7. Grading the Passages
7.1. Keeping track of the time
If you want to keep track of the time spent on each passage using a specific method (e.g., analytical method or holistic method or both), click on the Start button on the right top corner right before you start to grade a passage. When you are finished with that passage, put your cursor right behind Duration: at the end of the passage and click on the End button. The number shown is in seconds.
7.2. Customizing the Grading Category Lists
To use the analytical method, one needs an error category list. The list distributed with this program is for grading translations (especially English-Chinese translations). It is based on the ATA (American Translation Association) error category list.
To customize this list, click on 
(Project settings) at the top left corner of the main interface, and you will see the following window:
You can add a row and enter a new category, its abbreviation, and points that can be deducted. Pay attention to the +/- sign before the points. Likewise, you can delete a row. Click OK to adopt your customized category list for the current grading. Click Save to save the error category list for next time use.
To customize the holistic category list, click on Load Hol and select Holistic.xml, the following window will show. As with the error category list, you can add a row or delete a row.
To shift to the error category list, click on Load Err and select ErrorCategory.xml.
7.3. Grading the Passages Using the Analytical Method
When the grader spots an error in the passage, select that segment using the mouse, choose an error category from the drop down list, and assign to it points to be deducted using the mouse. (Caution: if you use keyboard arrows instead of the mouse, wrong points will be assigned. ) The graded passage will be like this:
The marked segments are in red, surrounded by the category tags and preceded by the point(s) assigned.
If the grader has inadvertently marked a segment or wants to make changes, simply delete the error category markup tags and the points just like what you would do in WordPad or Microsoft Word. To change the color of the marked segment from red to black, select that segment and click on the Nomark button on the right. Of course, you can always use Ctrl+Z to undo the latest action.
7.4. Grading the Passages Using the Holistic Method
Put the cursor right behind Holistic Grade: at the end of the passage, select a holistic category from the drop down list, and assign points to it using the mouse. Then you can select another holistic category and assign points to it until you have exhausted all the categories. In the target file, it will be like:
Holistic Grade:Accuracy+7; Expression+8;
7.5 Saving the Target File
The target file will be saved automatically every 5 minutes. You can click on the Save button on the right anytime you like to save the target file.
8. Calculating the Scores
When the grading is finished, click on 
at the top left corner of the main interface. The following window will appear:
In this table, the columns include text serial number, grading time, analytical score, and holistic scores (including accuracy and expression in this case). To export this table to Microsoft Excel, click on the Export button on the right.
The Basic Info drop down list includes two items: Overall Info and Errors Info. To shift to Errors Info, select it from the drop down list and click the Show button.
This table includes all the text segments that contain errors, the error categories and points deducted. To export this table to Microsoft Excel, click on the Export button on the right.
In Excel, you can sort the data, do calculations, and make data charts. The sky's the limit.
9. Contact Us
If you have any questions or suggestions, please don’t hesitate to contact us.
Sanjun Sun (sunsanjun AT gmail.com, Kent State University, Ohio)
Ruiliang Guo (fzygrl AT bift.edu.cn, Beijing Institute of Fashion Technology)