by Lyne des Roberts alias La Dame dragon
That’s it!… The verdict came in Wednesday (quite fast actually!)… Of course Dave could have pushed the issue further, but the PPC would have dragged the process for months. The question was: was he ready to invest more effort and energy into something that would probably totally drain him and leave him without any gain at the end? I would have myself chosen to pursue only for my very own satisfaction of having shaken the tree… but, it is not about me! It is about a man who can no longer be under such stress without seeing his condition worsen. And, since he will have to take his written test (the current 65 question version) again, he has to think of that stress first! He has not decided yet whether he will re-write his reading test (he already got his required level, but we both know he can get a C) or not… that will be up to him.
The answer from the PPC was really disappointing… Actually it did not answer the requests that were made: It was a long reasoning on the benefits of converting raw scores into standard scores… apparently, in some instances, candidates would find “standard scores easier to interpret because the numerical values for the mean and standard deviation are consistent across test versions”. In other instances, candidates would prefer “raw scores based on familiarity with these values”. What do you prefer? I do not need your answer because I know for sure you prefer to know the exact number of questions you got right!!!
When David told me he had obtained 42/80, I was quite proud of him because he had answered more than half of the questions correctly… At the time, we were not aware that both, the cut-off scores and his score, were standard! At least, the PPC was kind enough to provide him with his raw score which is 36/80! That changes everything! He did not get half of the answers correctly… as his trainer, it is what I need to know because it is the only way to assess his actual level and try to figure out what his weaknesses are (of course, if he had access to the test he wrote, it would certainly help! but in our dreams! that will never happen!). He was also provided with both, the raw cut-off scores and the standard cut-off scores:
- Level A Raw 27 Standard 36
- Level B Raw 41 Standard 46
- Level C Raw 58 Standard 57
Apparently the fact that the scores would be reported as standard scores was publicly communicated in advance on the PPC web site (given someone can find the said web site, which is incorporated in the Public Service Commission web site!). How come then Jenn, Janet and ZZ who had done thorough research prior to their tests did not know (and still do not!) that their results were actually not reflecting the exact number of correct anwers? Also they never questioned the fact they had not received any official signed document stating their levels, assuming it was part of the new procedure… Dave searched in the PPC archives to find such a public announcement: all he could find was a memo sent to assessors to tell them the PPC would do the conversion into standard scores (therefore they had to forward all candidates’ answer sheets to the PPC) in order to avoid any mistakes. Average John/Jane Doe would not even understand the content of such a message anyways, thinking it is addressed to specialists and has nothing to do with the tests themselves. Usually, when communicating news or modifications publicly, we use plain English to make sure everybody will understand… not some internal statistical jargon… Anyways…
Before all of this, although I always thought the Commission’s testing tools were questionable (once I met a guy who had obtained an E – exemption – in reading and a X – no knowledge at all – in oral interaction… he admitted he knew nothing, but had always been very lucky at lottery and multiple choice exams!) and seemed to be focused on figures (for mere statistical purposes I guess!) rather than on knowledge of the second language, I had never contemplated the possibility of finding out flaws such as the ones I discovered while researching for Dave’s appeal. All I can say is that all this is very sad… and even more sad because the PPC will always come up with some rationale or reasoning that actually no one (unless they are experts in the field) can argue.
What will be next? Well… since Dave’s new accommodations will take long, his training will be extended (again) until he will re-write his tests.. and since his raw score brought his weaknesses to light, I will have him review some grammatical basics (I do think we will have plenty of time for doing so…)… all this will postpone his oral test and I will not be able to focus on that until he gets his required level in writing. What a waste of time and money!
I realized the machine is way too big to try and win a battle against it… however, it is every single civil servant’s right to ask questions and request more information when they think their rights have been encroached upon. It is a matter of self-respect and integrity… and, perhaps, if there are enough people out there to start asking questions, the Commission and the PPC will eventually show more transparency and communicate better with the people they are assessing. If nothing else, Dave won something worthy… he standed up for his rights and found the courage to rock the boat because he knew he was right… In my book, he is a true winner!!!
This blog moved here, if you are ever interested to read more on those issues…
And… to whom it may concern:
The cut-off scores for the new 65 question written test, in effect since June 2, 2008, are expressed in RAW scores, as well as the obtained scores!… 8)
B = 33, C = 47, E = 57
The cut-off scores for the reading test remained unchanged and are also expressed in RAW scores, as well as the obtained scores!…
B = 38, C = 51, E = 59