The US Army is responsible for recruiting for its Special Operations Command, a unit of highly specialized personnel responsible for challenging and critical missions, where its members come from various branches in the larger military (Army, Navy, and Air Force). The recruiting and hiring process is very time-consuming, and the risks associated with hiring an unsuccessful candidate are very high. The standard hiring pipeline is comprised of recruiting the personnel, assessing the completed applications, and selecting and retaining the hires. To improve the current hiring processes, they entrusted us with data spanning from 2011 to 2021, over 30,000 records (or candidates) of application packets. After analysing the data, we found that many of their key issues lied within the very first step of the hiring process: recruitment. Within the recruitment step, only 1/3 of applicants actually submitted their applications, while 2/3 (20K service members) did not submit their applications despite being interested and actually completing various questions. In tandem with application completion, another equally important finding was the amount of missing data, where various question categories numbered above 50% missingness, and the inconsistent date formatting across the data, regardless of whether the applicant submitted the application or not.
In an effort to mitigate this missingness to bolster future analytical efforts (including modeling outcomes) and getting more applications into the assessment pool, we opted for three key recommendations, substantiated by what we observed in the data. For the former, enforcing data validity, such as reasonable date ranges, and optimizing data collection, such as requiring the completions of key questions before moving on to other sections of the application, are the recommendations to generate trustworthy data with less missingness. In order to monitor the progress of an application on an applicant-to-applicant basis, we recommend adding logging mechanisms to learn which categories detain applicants, since we have no current way to measure the order of the questions. Lastly, as methods to directly improve application submission rates, we recommend applying A/B tests to test ideas before moving them into production. One of the suggested experiments is e-mail nudging applicants who have made it past a certain question completion threshold to motivate them to finish the application, because we observed a specific group of applicants that could have submitted given the similarity of their question completion rates to applicants who did submit. As for the other suggested experiment, we recommend an A/B test where groups of applicants receive the original or shortened versions of the applications, to test if application length is stopping applicants from submitting, given that the application length is currently 100+ questions.
