|
|
Autogen for the Army Research Institute for Behavioral Sciences
Project Description
ARI's Occupational Analysis Office, in cooperation with U.S. ARMY TRADOC, desired to survey students at
Army Schools for effectiveness of course curriculum. The process was to be automated, with
browser-based electronic data collection for multiple surveys. They were interested in two approaches
for analyzing
student learning:
- Job Analysis (Task Analysis): The relevance of Army School
curriculum, based on
- Quantity of task performance,
- Emphasis of task training and
- Difficulty of performing tasks to standard.
- External Training Evaluation: how well were students trained, based on
- Post Student evaluation
- Post Supervisor evaluation
In addition, this software was required to account for:
- All Army Schools
- All possible Ranks of respondents
- All MOS's
- All skill levels and specialty courses (ASI's)
- All possible Duty Positions
- Creating New Optional Questions
In 2001, a new approach to data analysis (AOT Analysis) was requested to solve the following
problems:
- Identify tasks that are similar between respondents based on Percent Doing and Frequency of
task performance.
- Automatically separate respondents into distinct groups based on co-performance.
- Manually separate respondents into distinct groups based on Unit populations.
Raosoft Inc. Software Solution
With our point of contact in the Occupational Analysis Office, questionnaire survey shells were created
by using the Raosoft InterForm program. The shells were created to
account for all possible MOS's, ranks, and skill levels for the Job
Analysis (Task Analysis) and External Evaluation
paradigms.
Based on Raosoft InterForm, these shell files were created with hooks into the Autogen program. Autogen
was created as a specialized integrated application to be sent to each Army School for evaluation
training. This program
contains the following components and is easy enough for novices to use:
- Create Survey: Survey administrators add local content to survey shells.
- Data Collection: Create survey browser-based distribution files in the following format:
- Windows .exe files
- Windows web server format (.cgi)
- Append Data: Combine collected data from the various distribution methods.
- Database Tools: View project data in spreadsheet format and export data to other text and
SPSS format.
- Create Reports: Create Standard or Custom Reports.
- AOT Analysis: Create the various high level reports based on statistical analysis and
mathematical models.
- An extensive HELP file.
- Tutorials for creating a Job Analysis and External Evaluation survey.
In addition, an administration application for the Occupational Analysis Office was developed to create
and
edit the following information:
- The list of Army School Names to be distributed with Autogen
- The instruction content for the separate modules in Autogen.
- The list survey shells applicable for each MOS based on Enlisted, Officers and Warrant
Officers.
- The list of ASI names and descriptions.
- The separate Question Libraries for Job Analysis and External Evaluation.
Benefits to Improve Productivity
With Autogen, ARI's Occupational Analysis Office now has a single application to distribute to all Army
schools to analyze the effectiveness of the Army curriculum. Each Army school can input local
information,
distribute Job Analysis (Task Analysis) and Training Evaluation surveys
and combine the data when collected.
Distribution of surveys can be through
web surveys, by email, by diskette, or over the LAN. They can produce standard or create custom
reports or create high level statistical reports based on mathematical modeling.
|
