Most of you are probably aware of the GRE or Graduate Record Examination. Those applying for graduate programs are required to report scores from this standardized test. The GRE, along with the resume/curriculum vitae illustrating depth and breadth of experience, GPA, letters of recommendation, and an essay are evaluated for acceptance into a graduate program. Although the weight given to the applicant’s GRE varies among institutions, nearly all schools require the GRE and set minimum scores for application/acceptance. Typically top institutions and programs institute a high minimum GRE score to reduce the number of applicants and to ensure highly accomplished applicant pools. However, to what extent does the GRE predict success in graduate school? Does the GRE accurately measure an applicant’s ability to synthesize and apply knowledge, acquire and assimilate new information, or familiarity with mathematics, vocabulary, biology, history, chemistry, etc.?
Critics of the GRE, myself included, feel that the GRE does not evaluate any of these but simply provides a metric for how well a person can master GRE test taking procedures. ETS, the nonprofit that administers the test, states “the tests are intended to measure a portion of the individual characteristics that are important for graduate study: reasoning skills, critical thinking, and the ability to communicate effectively in writing in the General Test, and discipline-specific content knowledge through the Subject Test” However, the Princeton Review guide states, “ETS has long claimed that one cannont be coached to do better on its tests. If the GRE were indeed a test of intelligence, then that would be true. But the GRE is NOT [emphasis original] a measure of intelligence; it’s a test of how well you handle standardized tests.” Indeed, training for the test, either in a GRE course or with a book, involves more learning test taking strategies rather than a focus on acquirement or application of specific knowledge, i.e. “cracking the system”. Before the days of computerized exams, more math questions were included on the test than the average mathematically literate undergraduate could solve in the designated time period. To prep for the math section of the GRE, I, and others, learned not geometry or algebra, or even logic, but rather methods, i.e. cheats or tricks, to quickly move through the question. Of course, these tricks provide little help when math is encountered in either academia or the real world.
Who evaluates and writes the questions on the GRE? The Princeton Review states “You might be surprised to learn that the GRE isn’t written by distinguished professors, renowned scholars, or graduate school admissions officers. For the most part, it’s written by ordinary ETS employees, sometimes with freelance help from local graduate students.” Unfortunately, I am not able to retrieve information about question writers online either from ETS or other sources, so there is little I can comment on. However, the lack of transparency and accessible information should give all of us pause.
I am from the old guard and took my GRE with a No. 2 pencil with which I spent hours incessantly filling little circles. Modern tests are computerized and use a computer-adaptive methodology. Simply, the difficulty of questions is automatically adjusted as the test taker correctly or incorrectly answers questions. A complicated formula is used based on the level of questions and how many you answered correctly to determine your “real GRE score”. There is substantial criticism of the computer-adaptive methodology. One, if a test taker suddenly encounters an easy question mid-exam, they may deduce they have not been performing well thereby affecting their performance through the rest of the exam. Test takers may also be discouraged if relatively difficult questions are presented earlier on. Second, unequal weighting is given to questions, with earlier questions receiving more and setting precedent for later questions, biasing against test takers who become more comfortable as the test continues. ETS is aware of these issues. In 2006, they announced plans to radically redesign the test structure but later announced “Plans for launching an entirely new test all at once were dropped, and ETS decided to introduce new question types and improvements gradually over time.”
So who is ETS? They are a nonprofit 501(c)(3) created in 1947, located in Princeton, New Jersey. In an article from Business Week…
”Mention the Educational Testing Service, and most people think of the SAT, the Scholastic Aptitude Test that millions of high schoolers sweat over each year in hopes of lofty scores to help ease their way into colleges. But if ETS President Kurt M. Landgraf has his way, Americans will encounter the testing giant’s exams throughout grade school and right through their professional careers. Landgraf, a former DuPont executive brought to the organization in 2000 to give ETS a dose of business-world smarts, has a grand vision for the cerebral Princeton (N.J.) nonprofit. Worried that the backlash against college testing means a lackluster future for the SAT and other higher-ed ETS exams, Landgraf has been trying to diversify into two growth markets: tests and curriculum development for grade schools, where the federal No Child Left Behind Act has spurred national demand for testing, and the corporate market, where Landgraf sees potential growth in testing for managerial skills. By 2008, he hopes expansion in these two areas will more than double ETS’s $900 million anticipated 2004 revenue. “My job is to diversify ETS so we are no longer reliant on one or two major tests,” he says.
Does this sound like language applied to a nonprofit? From the New York Times…
It has quietly grown into a multinational operation, complete with for-profit subsidiaries, a reserve fund of $91 million, and revenue last year of $411 million… Its lush 360-acre property is dotted with low, tasteful brick buildings, tennis courts, a swimming pool, a private hotel and an impressive home where its president lives rent free… E.T.S. has come under fire not only for its failure to address increased incidents of cheating and fraud, but also for what its critics say is its transformation into a highly competitive business operation that is as much multinational monopoly as nonprofit institution, one capable of charging hefty fees, laying off workers and using sharp elbows in competing against rivals… ”E.T.S. is standing on the cusp of deciding whether it is an education institution or a commercial institution,” said Winton H. Manning, a former E.T.S. senior vice president who retired two years ago. ”I’m disappointed in the direction they have taken away from education and public service.”
In response to growing criticism of its monopoly, New York state passed the Educational Testing Act, a disclosure law which required ETS to make available certain test questions and graded answer sheets to students.
For all practical purposes ETS has grown into a for profit institution trading on its nonprofit status to create a monopoly (read the New York Times and Business Week articles for more alarming revelations). For example, Duke had 8,303 graduate applications for the fall of 2009, at $190 per test that is $1,577,570 for just one school for one year. From all schools, ETS in 2007 pulled in $880 million, and $94 million in profit after expenses. ETS also markets through one of their subsidiaries, and for profit, a test book at $23.10 (at Amazon). “We prepare the tests-let us help prepare you!”
But what of the original question? To what extent does the GRE predict success in graduate school? A meta-analysis in 2001 by Kuncel et al. demonstrated that correlations between GRE scores and multiple metrics of graduate performance were low. Correlation with graduate GPA ranged from 0.34-0.36. With performance as evaluated by departmental faculty the correlation ranged from 0.35-0.42, time of degree completion ranged from -0.08-0.28, citation count ranged from 0.17-0.24, and degree attainment from 0.11 to 0.20. While encouraging these correlations are positive (in most cases) and even accounting that GRE is supposed to be evaluated in the context of other materials (but often are not), these correlations are not that impressive. Do we need the GRE scores to evaluate applicants? Interesting, the same study also demonstrated that undergraduate GPA performed equally well as the GRE. Why then do we need $190 test loaded with faults and biases when GPA is sufficient?
I end by saying good luck to my wife who faces the GRE this fall.
Great article. One of the main and many reasons why I’ve been avoiding grad school is because of the GREs. I am an horrible test taker and when I heard that the GREs took out the analytical part and replaced it with a written one sometime ago, I knew I was screwed. They took away my best part and replaced it with my worse. I got into undergrad somehow without them receiving my SATs so I just hope that might happen again.
I agree. Great article. I am one of the fortunate few who takes standardized tests with no problem, but my wife is not. She agonized over the GRE for two month prior to taking it and then she did fairly well but nowhere near what her intelligence and grades in previous schools would have indicated. The GRE is just another way for ETS to make a ton of money, which as a 501(c)(3) they should not be focused on profit. I think that it may be time for the IRS to review their tax-exempt status. I will tackle the GRE this fall also and am scared to death….
What’s the big deal? The GRE is just as easy as the SAT! They’re both pretty basic in what they require… and that goes for the subject specific ones as well (at least the Biology one which I took). I only had a week to prepare and found it relaxing!
For those of us having studied overseas and wanting to enter US graduate programs etc., the GRE is much more reliable than GPAs! Basically because in most countries and universities outside of the US it’s nearly impossible to find anyone with a GPA above 3. In Spain having a GPA of 2 is considered a decent grade! Grading is much tougher over here in Europe. The only exam you’re likely to get 100% on is a multiple choice, and that isn’t at all easy! Anything written will almost never get you a perfect grade because there’s almost always something you could have written more…
Thanks for sharing the results of the quantitative research. I’d like to suggest an alternate explanation of the facts (but by no means a true one) : if GPA and GRE scores are equally good indicators, and if they’re both by themselves not particularly strong ones, might it then make sense to use both? And, for that matter, to use other, strictly qualitative predictors (recommendations, writing sample, personal statement, etc.)?
I’d guess (and might bet a small sum) this is the reason graduate schools still use the GRE. It’s in their interests to predict student performance as best as they can, and to junk indicators that don’t mean much. (Note, for instance, the disappearance of the GRE History exam, which was little but random trivia.)
You evaluation of the test, too, would be more convincing if you gave weight to an additional tutoring company. Princeton Review was founded in opposition to “buckle-down-and-learn-your-math-concepts” Kaplan, and champions a “these tests are crap — we’ll show you how to crack ’em without making you smarter!” mentality. Such a perspective certainly captures a piece of the truth, but not the entire thing. “There are more things in heaven and earth than are dreamt of in Princeton’s Review’s educational philosophy.”
I make no claims, though, to objectivity: I’m an independent GRE tutor. For anyone taking the test, I recommend he/she considers it an opportunity to re-master the (now-evaporated) fundamental math skills, radically expand vocabulary, and improve in analytical reading. If anyone’s interested, I’m happy to give book suggestions!
Good article. I’m one of those unlucky ones that does poorly on standardized tests. Although I’m an honors student in finance, with an in major GPA of 4.0 at a well regarded school (my overall GPA is 3.94), I did quite poorly on the quantitative portion of the GRE (about 33rd percentile). The problem with the test, as I see it, is that the quantitative section focuses on, and requires remarkable speed in, middle/high school math. So, as someone who is on the theoretical side of statistics & calculus, and has been taught as an undergrad to value accuracy over speed, the test is counter intuitive and doesn’t actually measure any of the quantitative skills needed for my chosen vocation.
The second major problem with the GRE is that rather than score, a applicant’s percentile placement is often looked at. Because the test is HIGHLY rigid in format and EXTREMELY coachable, particularly on the quantitative side, there is s significant advantage to those test takers who have the resources to pay for tutoring.
My worst fears have been confirmed through this article. It is not that i fear to do GRE at the end of the day i will do it for it is a requirement for United States Universities. But after going through the verbal sections of the old and new GRE, i have come to realise that this is not a good way of measuring capability of handling graduate work. Instead the exam throws one into a frenzy of claming vocabularies and familiarizng with the format which at the end of the day beats the logic of the initial intention.
The verbal part of the new GRE is even worse where by in the part of completing some blanks in a text, one can end up earning no credit because he answered two blanks correctly and failed one. Further more some questions can have two possible answers and the idea of the best is too subjective.
The exam is also unfair for people coming from other education systems that rarely use multiple choice questions…especially the common wealth. The last time i did multiple choice questions was more than 20 years while in primary school.
I have to wonder what the author of this great article does after putting out such vital information. Just hope he has not resigned to rest on his laurels where this issue is concerned. Crony capitalism and rampant corruption comes to mind after reading this article, not unlike the putrefying stench emanating from wall street and their cheaply bought politicians. Those connected to the GRE are thinking people, so it is hard to imagine advanced thinkers allowing such a rotten system to go on in perpetuity. I’ve already completed my masters at a state university but my harrowing experience with the GRE left me a bad taste and disappointed at how much advanced thinkers willingly allow themselves to be used.
Now ETS offers you the option of paying them even MORE money to hide your lowest test scores (if you pay to take it multiple times) from schools you are applying to. I hate ETS with an unyielding passion.
Hi Brandon, I know this is kind of an old thread but would appreciate it if you could give me any suggestions on books, materials, etc. I know the format has changed to the ‘adaptability’ of the test taker with the computer and was wondering if coaching is actually beneficial for the GRE. I am not a great standardized test-taker. I like to analyze and take my own time with problem solving. Do you think it is worth investing in Princeton Review or Kaplan coaching sessions? Which of the two would you recommend?
Thanks,
Zen
The GRE Biology subject test may very well not be much harder than the SAT2 biology subject test but the GRE Physics subject test covers questions from all areas of undergraduate coursework. It contains more questions than the average well-qualified physics undergraduate can handle; I have studied for it myself and I found many questions way too hard to answer all of them in the timeframe.
The GRE is good for measuring one thing other than how well one takes tests, and it is the ability of the test-taker’s nerves not to crack under pressure. That said, East Asian students (namely Indian, Chinese, Korean as well as Japanese students, and perhaps some from other East Asian countries to a lesser extent) typically score high on the GRE while being more likely to flounder in graduate school than students from other countries, presumably since they are practically trained since birth to ace standardized tests.
Case in point: the physics department at Vanderbilt (when Rob Knop worked there). Vanderbilt’s admissions committee separated the applications in three piles then: the locals, the Chinese and everyone else. The Chinese mostly came with high PGRE and GGRE scores, yet there were huge discrepancies between their GRE scores and what they actually achieved.
Take it from an award-winning, perfect 4.0 student; a statistics teaching assistant (after earning a 103% in the class), and not just the top score on ALL exams in my calculus class and top final grade, but the top in two entire calculus classes combined. I am receiving the highest honors at graduation – I even get to sit on stage with the “distinguished guests.” I tutor community college students in College Algebra. So, when you mention “now-evaporated, fundamental math skills,” you clearly speak from one who profits as a GRE tutor. The GRE quantitative section is akin to a foreign language. It is a racket, and these scores do not even remotely reflect one’s potential success in graduate school. Clearly, this test is designed to make it essential for students to pay $185 over and over again – or to pay tutors, such as yourself, to teach topics never addressed in school. What is the units digit of 18 to the 47th power? Should we not cave into expensive tutoring and pay to take the test multiple times, none of us could answer this question (in less than 1 minute 30 seconds based upon past schooling). Racket.