Reducing Information Asymmetries in the Youth Labor Market of Jordan with Psychometrics and Skill Based Tests

Abstract

Jordan's labor market for educated youth is characterized by high levels of unemployment, long periods of job search, and firms complaining that youth often lack the appropriate interpersonal and work skills. Search and matching theory offers a potential explanation: if education systems are such that graduates find it difficult to signal competence and achievement through grades and the quality of their institution, then employers might have difficult matching with suitable candidates, resulting in high unemployment. We developed and tested a labor market screening and matching service in Amman, Jordan, which aimed to generate higher employment for educated youth by reducing these matching frictions. This paper examines the first step in this process, which involved testing unemployed, tertiary-educated, youth on mental ability, English proficiency, soft skills, Excel ability, and also measuring their big five personality traits. We show that these measures have predictive power for subsequent employment and for earnings conditional on employment, even after conditioning on major, university, and other controls. Psychometric testing therefore offers the potential to reduce information asymmetries that result in labor market matching frictions.

Introduction

In common with a number of countries in the Middle East and North Africa, Jordan's labor market for educated youth is characterized by high levels of unemployment, long periods of job search, and firms complaining that youth often lack the appropriate interpersonal and work skills for the job (Angel-Urdinola et al. 2010; Almeida et al. 2012). In 2010, unemployment rates for men and women between the ages of 22 and 26 with a post-secondary degree were 19 percent and 47 percent, respectively. In a 2011 survey we conducted of 2000 firms in Amman looking to hire workers, 60 percent of firms claimed to have difficulty distinguishing between good and bad job candidates, and 64 percent said they had difficulty finding competent graduates.

One explanation for these patterns is offered by the search and matching theory of unemployment pioneered by Peter Diamond, Dale Mortensen and Christopher Pissarides.¹ In their model, unemployment can persist in equilibrium as costly search frictions make it difficult for jobless workers searching for work to match with firms with vacancies looking for workers. We might expect these search frictions to be larger in regions where education systems are less able to signal competence and achievement through grades and the quality of the institution. Improvements in the matching technology can then potentially reduce unemployment directly (through making it easier for firms to fill existing vacancies) as well as indirectly (by lowering hiring costs and thereby encouraging firms to create more vacancies). There is a large theoretical and macroeconomic literature which examines the role of search frictions in unemployment² but far less that examines at a microeconomic level the role of policy efforts to reduce these frictions. These frictions can be particularly severe for young workers, who lack work experience that can be used as a signal of worker quality.

We developed and tested a labor market matching service in Amman, Jordan, which aimed to generate higher employment for educated youth by reducing these matching frictions. Unemployed applicants were administered a series of tests to evaluate mental ability (verbal, quantitative, and spatial), technical ability in Excel, fluency in English, soft skills, and personality type.

In order for this approach to improve employment outcomes, it must first be the case that these testable attributes of individuals add value for predicting employment beyond what can be easily observed from a curriculum vitae (CV). In this short paper, we focus on establishing this first step. We show that our tests do have predictive power for the likelihood female graduates have found employment 10 months later, even conditioning on baseline observables, and strong predictive power for the salaries that males and female earn conditional on working for both male and female graduates. Psychometric testing therefore offers the potential to reduce information asymmetries that result in labor market matching frictions.

The Study Population

We launched the pilot in January 2012 and restricted eligibility to Jordanians who had graduated from community college or university after May 2009. The screening and matching services were offered to participants for free, and participants were recruited through advertisements in local newspapers, radio stations, cafes, Facebook, and a telephone campaign to recent graduates from 14 universities and community colleges in and around Amman. In total, 1567 recent graduates were recruited and participated in the pilot between December 2011 and December 2012.

Appendix Table 1 summarizes some basic characteristics of the applicants: 58 percent are female, 80 percent are university graduates and 20 percent community college graduates, and the average age is 23 years. Students have a mix of majors, with accounting and business, engineering, and computing and information technology the most common. Nearly all were unemployed at the time they took our assessments, but 55 percent of the females and 74 percent of the males had some previous work experience.

Table 1.

Do Psychometrics Help Predict Employment 10 months later?

	Males			Females
Mental Ability	0.0274	0.0168	0.0154	0.0344	0.0414*	0.0450*
	(0.0241)	(0.0252)	(0.0264)	(0.0213)	(0.0227)	(0.0239)
Soft Skills	0.0202	0.0208	0.0508**	0.00951	0.0124	0.0205
	(0.0223)	(0.0238)	(0.0259)	(0.0202)	(0.0220)	(0.0225)
Excel Test	0.0309	0.0204	0.0409	0.0148	−0.00642	−0.0265
	(0.0212)	(0.0224)	(0.0252)	(0.0203)	(0.0222)	(0.0253)
English Test	0.0108	−0.000867	−0.000448	0.0827***	0.0458*	0.0586*
	(0.0249)	(0.0283)	(0.0321)	(0.0234)	(0.0272)	(0.0300)
Analytical Personality Score	−0.00461	−0.00678	−0.0242	−0.00909	−0.00854	−0.0181
	(0.0282)	(0.0288)	(0.0311)	(0.0263)	(0.0276)	(0.0286)
Emotional Personality Score	−0.0127	−0.0171	−0.00962	−0.0469*	−0.0417	−0.0388
	(0.0274)	(0.0284)	(0.0306)	(0.0251)	(0.0266)	(0.0277)
Extroverted Personality Score	0.0370	0.0484*	0.0494*	−0.0621***	−0.0670***	−0.0648**
	(0.0255)	(0.0261)	(0.0267)	(0.0241)	(0.0256)	(0.0268)
Opportunistic Personality Score	0.0254	0.0173	0.0422	0.0352	0.0301	0.0311
	(0.0238)	(0.0246)	(0.0271)	(0.0224)	(0.0232)	(0.0247)
Dependable Personality Score	−0.00568	0.0101	−0.00881	0.0501**	0.0613**	0.0650**
	(0.0257)	(0.0269)	(0.0282)	(0.0249)	(0.0257)	(0.0270)
Observations	546	517	511	745	706	705
Baseline variable controls	no	yes	yes	no	yes	yes
Controls for major and university	no	no	yes	no	no	yes
P-values for testing:
All 4 tests jointly zero	0.126	0.620	0.070	0.000	0.034	0.011
All 5 personality traits jointly zero	0.483	0.314	0.172	0.010	0.012	0.014
Mean Employment Rate	0.647	0.642	0.640	0.495	0.503	0.502

Notes: Coefficients are marginal effects from probit estimation. Robust standard errors in parentheses.

*, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively.

Baseline control variables are tawjihi score, university vs community college dummy, years since graduation, whether they have ever worked before, and whether they are unmarried.

Psychometric Measurement

Each program participant attended a one-day employment screening session at the Business Development Center (BDC), a leading employment training services provider in Jordan. During this session, program participants completed a series of job skill tests and psychometric assessments designed and validated by Dr. Marwan Al-Zoubi, a psychology professor at the University of Jordan who specializes in organizational behavior and work psychology. The tests and assessments were administered by computer except for the soft skills test; the tests include the following:

1. Mental Ability: A timed test consisting of 45 questions, equally divided between verbal, quantitative, and spatial reasoning. We form a principal component to aggregate scores from these categories into a single ability index.

2. English Proficiency: A timed test consisting of 15 vocabulary and grammar questions, 15 reading comprehension questions, and 20 listening based questions. The scores are then normalized to a score out of 100 based on comparisons to the performance of University of Jordan students on the same test prior to the launch of the pilot.

3. Excel Proficiency: A timed test that measures the participants' ability to write text in cells, add and delete rows and columns, sum variables, and calculate the mean of a group of scores. The score is calculated based on the participants' ability to complete 17 Excel tasks correctly, and the score is normalized to a score out of 100.

4. Soft Skills: Soft skills were measured by three interactive exercises. The first was based on a group exercise, in which five to eight participants were put in a group and tasked to redesign a failing amusement park in Jordan. They were each given a predefined role and evaluated on how they work in groups. The second exercise was a role-playing game designed to test the participant under pressure. The participant plays the role of a customer service associate, and the evaluator, an angry customer who had purchased a computer that broke down. The participant's goal is to calm the customer and come up with a solution within the framework of the company's rules. The final assessment was a skills-based interview where the soft skills specialist asks questions to elicit examples of leadership, teamwork, and overcoming obstacles. The appendix provides additional details. We form a principal component analysis of scores in the 10 soft skill categories to form a single soft skills index.

5. Big-5 Personality Traits: Personality traits were measured through a series of 300 questions assessing the following five characteristics: analytics, extraversion, emotions, opportunism, and dependability (Paunonem and Jackson 1996). These were translated into Arabic and validated on a sample of students at University of Jordan (Al-Zoubi 2014). We calculated the Big-5 personality traits as the mean of their subcharacteristics, and we normalized the Big-5 personality traits for ease of interpretation.

Do these Measures Predict Employment and Earnings?

In order for these skill tests and psychometric assessments to be useful in reducing incomplete information and hence search costs for employers, we need them to contain additional information that is useful for determining employment beyond the easily verifiable background information of contained in job candidates' CV. Ideally, we would evaluate these tests and assessments on objective measures of labor productivity, but the heterogeneity of jobs in the labor market make this ideal approach infeasible. Instead, we investigate the extent to which test and assessment scores are predictive of subsequent employment and earnings.

We use a follow-up survey of program participants conducted in May 2013, 16 months after the program was launched, and on average ten months after the tests. The follow-up survey re-interviewed 1291 applicants, for an attrition rate of 17.7 percent.³ At the time of the follow-up survey, 49.5 percent of the females in our sample and 64.7 percent of the males were employed. Mean monthly earnings conditional on employment were 324 JD (US$459) for females and 378 JD (US$536) for males.

As a first step to evaluating skill tests and psychometrics predictability of labor productivity, figure 1 examines the bivariate associations between four test scores and employment outcomes. Consider first employment. We see a steep positive linear relationship between employment and English proficiency, and between employment and mental ability, for females, with much less of a relationship with these characteristics for males. There is a flatter but still positive relationship of soft skills with employment for both genders, and no relationship with Excel scores. Turning to earnings conditional on working, English proficiency, mental ability, and soft skills all have positive associations with the amount earned for females, and English and soft skills have positive associations with the amount earning for males.

Figure 1.

Bivariate Relationships between Test Measures and Subsequent Employment and Earnings (Females shown by solid black lines, males by black dashed lines, 95% confidence intervals in gray)

Note: Curves plotted between the 10^th and 90^th percentiles of the test score distributions.

Next, we examine whether these associations continue to hold when examined jointly with the other test results and personality characteristics, before controlling for key observable characteristics that might be observed through a CV. For ease of comparison, we standardize the test scores and personality statistics as z-scores – a one unit change then represents the impact of a one-standard deviation (SD) change in that variable.

Table 1 presents the results from probit estimation of the likelihood of being employed. The first column for each gender just contains the test scores and personality characteristics as controls, the second column adds additional observables as controls.⁴ The third column then adds additional controls for major (21 dummies), and for tertiary institution (15 dummies). Table 2 presents the same three specifications for a linear regression of monthly earnings conditional on being employed. The foot of both tables tests the null hypotheses that the coefficients on the 4 tests are jointly zero, and that the coefficients on the five personality traits are jointly zero.

Table 2.

Do Psychometrics Help Predict Earnings Conditional on Employment?

	Males			Females
Mental Ability	−0.321	3.716	7.979	9.324*	9.203*	11.28*
	(8.595)	(8.336)	(9.068)	(5.235)	(5.362)	(5.859)
Soft Skills	33.97***	29.63***	27.46***	17.14***	16.70***	13.60**
	(7.850)	(8.573)	(9.245)	(4.877)	(5.015)	(5.525)
Excel Test	−0.524	−0.725	2.036	4.160	2.862	−2.092
	(7.213)	(8.214)	(8.671)	(5.058)	(5.355)	(6.610)
English Test	20.69***	11.43	3.832	40.49***	35.33***	30.21***
	(7.771)	(9.172)	(10.72)	(6.545)	(7.897)	(8.101)
Analytical Personality Score	−14.76	−10.69	−4.556	6.105	6.191	5.599
	(9.918)	(9.578)	(9.888)	(6.116)	(6.193)	(6.432)
Emotional Personality Score	−1.608	−6.217	−4.084	−7.006	−5.642	−6.981
	(9.893)	(10.41)	(10.45)	(5.932)	(5.892)	(6.086)
Extroverted Personality Score	9.206	11.66	8.607	3.760	4.439	7.106
	(8.403)	(8.807)	(9.427)	(7.107)	(7.088)	(7.347)
Opportunistic Personality Score	−0.643	−4.365	−7.114	−3.911	−4.736	−7.128
	(8.538)	(9.023)	(8.657)	(6.499)	(6.580)	(6.342)
Dependable Personality Score	13.20	19.80**	22.45**	−2.973	−4.772	−4.808
	(8.892)	(9.438)	(9.549)	(7.019)	(7.605)	(7.584)
Observations	346	325	325	356	342	342
R-squared	0.103	0.126	0.228	0.242	0.272	0.392
Baseline variable controls	no	yes	yes	no	yes	yes
Controls for major and university	no	no	yes	no	no	yes
P-values for testing:
All 4 tests jointly zero	0.000	0.002	0.019	0.000	0.000	0.000
All 5 personality traits jointly zero	0.394	0.147	0.125	0.752	0.754	0.604
Mean Monthly Income conditional on working	378	379	379	324	324	324

Notes: Coefficients are regression estimates. Robust standard errors in parentheses.

*, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively.

Baseline control variables are tawjihi score, university vs community college dummy, years since graduation, whether they have ever worked before, and whether they are unmarried.

Table 1 shows that both our tests scores and the personality traits are predictive of employment for females, even conditioning on all of our controls. The strong relationship with English proficiency seen in figure 1 is statistically significant and continues to hold even after adding these additional controls. A 1 SD increase in the English test score is associated with a 5 to 8 percentage point increase in employment. The mental ability score is associated with a 3 to 4.5 percentage point increase in employment for females, while the Excel test and soft skills test have no statistically significant relationship. The dependable personality trait is statistically significant as a predictor of employment for females, which is consistent with the idea that punctuality and reliability are valued by employers. Extroversion has a negative association with employment for females, which might reflect a cultural preference of employers, especially since extroversion has a positive and significant association with employment for males. Overall, the different measures have far less predictability for male employment, although males with higher soft skills have higher likelihoods of being employed.

Table 2 shows that both our soft skill measure and our English proficiency score help predict the salaries these individuals earn once they are employed. A 1 SD increase in soft skills is associated with 27 JD higher monthly earnings for males (7% of mean earnings), and 14 JD higher monthly earnings for females (4% of mean earnings). English proficiency is no longer a statistically significant predictor of conditional earnings for males once other controls are added, but continues to be statistically significant for females: a 1 SD increase in English score is associated with 30 JD higher monthly earnings for females (9% of mean earnings). Mental ability also helps predict female earnings but not male earnings. In contrast, for neither males nor females can we reject that the 5 personality traits coefficients are jointly zero.

In addition to being statistically significant, the additional predictive power of these tests is also economically significant. For females, the R² of conditional earnings increases from 0.313 for a regression with baseline controls, major, and university to 0.392 for the specification in the last column of Table 2, a 25.2% increase in predictive power. For males, the R² increases from 0.175 to 0.228, a 30.5% increase in predictive power.

Conclusions

Taken together, these results suggest that our psychometric measures contain additional information about future employability beyond that which would be contained in a standard CV. In particular, our English proficiency, soft skills, and mental ability measures all have some predictive power for either employment and/or earnings. The predictive power appears to be stronger for young women than young men, which may be explained by the lower employment rates of women, perhaps reflecting higher degrees of selectivity by both young women and by employers as to whether young women work. In principle, our psychometric assessments contain information, which could reduce information asymmetries and help improve matching between employers and workers. The challenge, which we are examining in ongoing work, is to then whether this information can be successfully used to match job-seekers with firms and generate increased employment as a result.

1

A nice summary is provided in the advanced notes accompanying their joint Nobel Prize http://www.nobelprize.org/nobel_prizes/economic-sciences/laureates/2010/advanced-economicsciences2010.pdf [accessed May 1, 2014].

2

See the reviews by Petrongolo and Pissarides (2001) and McCall and McCall (2008).

3

Appendix Table 3 presents evidence that psychometric measures for males are not significantly correlated with attrition, but females with higher soft skills and more analytic personalities are more likely to respond. We discuss robustness to this in the appendix, and note it shouldn't affect our main conclusions.

4

These controls are for when they graduated, whether they went to university versus community college, whether they have ever worked before, their score on the end of high school national examination (tawjihi), and their marital status.

Appendix

Appendix Table 1 provides summary statistics on the participants in this study.

Appendix Table 1.

Baseline Summary Statistics of Participants

	Female Mean	Female SD	Male Mean	Male SD
Age	23.2	2.0	23.9	2.0
Years since Graduation	0.86	1.04	0.53	0.88
University	0.73	0.44	0.89	0.31
Ever Worked	0.55	0.50	0.74	0.44
Single	0.89	0.31	0.97	0.16
Tawjihi Score	75.68	11.31	70.72	10.51
Accounting or Business	0.31	0.46	0.39	0.49
Engineering	0.10	0.30	0.16	0.36
Computing or IT	0.10	0.30	0.15	0.35
Mental Ability Score	0.08	1.30	−0.11	1.31
Soft Skills Score	−0.31	2.30	0.43	2.38
Excel Test	60.60	22.41	69.92	19.16
English Test	58.24	19.90	56.88	19.43
Analytical Personality	6.68	2.94	6.83	2.90
Emotional Personality	3.53	3.31	3.64	3.22
Extroverted Personality	3.35	2.84	3.84	2.71
Opportunistic Personality	4.78	3.67	5.21	3.71
Dependable Personality	4.26	2.57	4.35	2.68
Sample Size	907		660

Further details on the soft skills scoring

The soft skills assessment is an interactive exercise judged by soft skills specialists based on a detailed rubric designed by Marwan Al-Zoubi. Soft skill specialists are former human resource managers who lead soft skills training for BDC's Maharat program. The detailed rubric encompasses 10 distinct categories of soft skills and is composed of five objective criteria for each category to be rated on a 10 point Likert scale. Two soft skill specialists evaluate each sub-category on a scale from 1 to 10. At the end of each day, the specialist compared their scores and reconciled differences by averaging the scores if there was a difference of 2 or less points per category. If there was a difference of more than 2, evaluators discussed the candidate and reached a compromise. Appendix Table 2 presents the soft skills scoring rubric.

Appendix 2.

Soft Skill Evaluation Rubric

1. Listening	6. Leadership
Faces speakers squarely	Active in the discussion and provides effective points
Adopts open posture	Expresses confidence in non-aggressive style
Leans toward speakers	Creates positive atmosphere
Maintains eye contact with speakers	Goal oriented
Does not cut off speakers	Tries to reach decisions
2. Responsiveness (Posing Questions)	7. Supportiveness
Asks follow up questions	Asks quieter people for their opinions
Asks open-ended questions	Tries to coordinate the discussion effectively
Asks non-leading questions	Solves conflict between group members in a diplomatic style
Asks clarification probes	Recognizes others' contributions
Asks questions in non-threatening style	Encourages others to provide more ideas
3. Presentation	8. Initiative
Speaks with a clear voice	Makes good impression on others
Maintains eye contact	Encourages the group to think about the future
Speaks concisely	Provides high quality ideas
Expresses thoughts through body language	Volunteers to do unwanted tasks
Smoothly transitions from one subject to another	Encourages group to achieve goals
4. Self-confidence	9. Organization
Speaks up	Manages time of the discussion effectively
Does not ramble	Reminds others about time limit
Defends opinions in non-aggressive style	Set up tasks for her self and committed to deliver it to the end
Delivering messages and opinions in a firm way	Ask others to go back to the core point when discussion go pointless
Asks speakers to give examples	Defined problems and set up certain steps to solve it
5. Influence	10. Teamwork
Presents arguments in various ways	Builds good relations with other group members
Seeks agreement from others	Tolerant with other team members and accepts their requests
Tries to direct discussion in non-aggressive style	Asks for information or opinions in an encouraging way
Provides opinions and clarifies how they will be beneficial for others	Gives information or opinions which indicate serious involvement in the task
Prepared to compromise to achieve mutual agreement	Works hard on the tasks and was fully involved

Two soft skill specialists evaluate each candidate on all 10 soft skill categories during the group discussion. The customer service role-playing game is assessed by one evaluator on soft skill components 1 through 5 in the table above. The skills-based interview was assessed by one soft skills specialist on the soft skill components 6 through 10 in the table above.

Survey Attrition and Psychometric Scores

Appendix table 3 examines the association between our psychometric measures and attrition in the follow-up survey. For males, although the English test is marginally significant once we control for major and university, we cannot reject the null hypothesis that our four tests jointly do not predict attrition (p = .12), and that the five personality traits jointly do not predict attrition (p = .99). For females we find soft skills, proficiency with Excel, and analytical personality traits to be positively correlated with survey response. To examine how sensitive our results are to this selective attrition, we consider a bounding exercise. If we assume that the individuals with high soft skills who answer our survey, but who wouldn't have answered if they had lower soft skills, were all employed, then dropping these individuals would still result in a near zero and statistically insignificant association of employment with soft skills for females. Conversely, if the additional responders are those who are all unemployed, we find a positive, and marginally significant association of female employment with soft skills, with a coefficient of 0.04 (p = .087) in the analog of the last column of Table 2. Selective survey response therefore may have a minor impact on whether we consider soft skills to be predictive of employment, but our main conclusions that employment and earnings are predicted in part by test scores appears robust to this small amount of selective attrition.

Appendix Table 3.

Do Psychometrics Predict Survey Attrition?

	Males			Females
Mental Ability	−0.0108	−0.0169	−0.0192	−0.00741	0.00317	0.00651
	(0.0176)	(0.0178)	(0.0171)	(0.0143)	(0.0145)	(0.0146)
Soft Skills	0.0263*	0.0312*	0.0238	0.0398***	0.0426***	0.0380***
	(0.0154)	(0.0160)	(0.0173)	(0.0134)	(0.0139)	(0.0139)
Excel Test	−0.0174	−0.00903	−0.0219	0.0240*	0.0326**	0.0349**
	(0.0156)	(0.0161)	(0.0168)	(0.0138)	(0.0139)	(0.0149)
English Test	−0.0261	9.78e-05	0.0386*	−0.00969	−0.00264	−0.0112
	(0.0181)	(0.0201)	(0.0214)	(0.0156)	(0.0177)	(0.0188)
Analytical Personality Score	0.00249	0.00437	0.00368	0.0318*	0.0357**	0.0316*
	(0.0197)	(0.0193)	(0.0202)	(0.0178)	(0.0177)	(0.0176)
Emotional Personality Score	−0.0235	−0.0124	−0.000277	−0.0151	−0.0226	−0.0194
	(0.0188)	(0.0187)	(0.0185)	(0.0172)	(0.0165)	(0.0165)
Extroverted Personality Score	−0.000422	−0.000268	−0.00319	−0.0325**	−0.0359**	−0.0304*
	(0.0183)	(0.0179)	(0.0176)	(0.0163)	(0.0166)	(0.0165)
Opportunistic Personality Score	0.0121	0.00984	0.00928	−0.00668	−0.00195	0.000623
	(0.0162)	(0.0165)	(0.0175)	(0.0149)	(0.0151)	(0.0152)
Dependable Personality Score	0.00142	−0.00837	−0.0123	0.00195	0.00464	0.00187
	(0.0183)	(0.0190)	(0.0201)	(0.0159)	(0.0161)	(0.0163)
Baseline variable controls	no	yes	yes	no	yes	yes
Controls for major and university	no	no	yes	no	no	yes
Observations	660	622	621	907	856	856
R-squared	0.015	0.036	0.102	0.021	0.042	0.081

Notes: Coefficients are regression estimates. Robust standard errors in parentheses.

*, **, and *** indicate significance at the 10, 5, and 1 percent levels respectively.

Baseline control variables are tawjihi score, university vs community college dummy, years since graduation, whether they have ever worked before, and whether they are unmarried.

Author notes

Matthew Groh is a data scientist at the World Bank. David McKenzie is a lead economist at the World Bank. Tara Vishwanath is a lead economist at the World Bank.

We thank the TFESSD trust fund, the Knowledge for Change (KCP) trust fund, and the Research Support Budget (RSB) of the World Bank for their generous support of this project; Dr Marwan Al-Zoubi for his collaboration on developing the psychometric measures used here; the Business Development Center (BDC) and Dajani Consulting for their work in project implementation; and Nandini Krishnan and Nour Shammout for their collaboration on related aspects of this project. All views expressed are those of the authors and need not represent the views of the World Bank.