Deep Learning-based Online Exam Proctoring – A Novel Paradigm

Since times immemorial, examinations have formed the bedrock of the interlinked spheres of education and employment. Ancient Egypt, Mesopotamia, India, and China had independently devised various learning systems as well as relevant processes for assessing what was being learned. Formal theoretical education in these ancient societies was limited to the elite and the affluent, whereas priests, craftspersons, and merchants had more hands-on, practical training. Similarly, in China, government officials were appointed only after clearing intense exams designed under the watchful supervision of Emperor Zhang (AD 75-88).

In modern times, the democratization of education has precipitated across the world the explosive growth of educational institutions of various forms and functions. Examinations have become a rite of passage. And an entire industry dedicated to training students and employment-seekers to ace various examinations has become commonplace across societies, be it Japan or USA, or India.

Role of Technology and the Emergence of Online Examinations

The advent of technology sparked the next development in this sector – examinations could now be online and taken from anywhere across the globe via the marvelous medium of the world wide web.

Vis-a-vis pen-and-paper examinations, online examinations are more convenient, cost-efficient, time-efficient, and secure from P2P cheating and other malpractices. However, these manifest their own set of difficulties –

1. Technological and infrastructure challenges
2. Inability to grade hands-on learning and collaborative projects
3. Inability to grade subjective/ interpretive answers
4. Technically advanced forms of cheating – use of Virtual Machines (VMs) to run test platforms, dual screens, concealed webcams, bluetooth speakers, etc.
5. Concerns regarding candidate privacy and storing recorded data – login location, biometrics, proctoring video and image logs – compliance with General Data Protection Regulation (GDPR) and similar legislations across geographies
6. Concerns regarding candidate safety against verbal and sexual harassment from proctors

Covid-19 Pandemic – Constructive Destruction of the Examination Ecosystem?

The near-unimaginable disruptions caused by the Covid-19 pandemic and concomitant restrictions on socialization and movement instantaneously catapulted the online examination ecosystem to breathtaking heights. As schools, colleges, and workplaces rushed headlong to online mode, the traditional pen-and-paper examination model was irredeemably dismantled.

Many institutions altogether cancelled examinations where they could, at least as a temporary arrangement. But simultaneously, a tremendous new era of online examinations dawned. And it involved millions of students and employment-seekers remotely taking exams from the comfort of their homes, away from the watchful eyes of invigilators.

Two problems underlying this epochal change quickly manifested –

1. It is impossible to manually proctor millions of candidates’ simultaneously, and
2. Granular, eagle-eyed supervision of candidates had to be balanced against digital privacy violation and proctorial harassment.

With this emerged the urgency of evolving automated proctoring systems. Such a system is not limited to merely recording and logging audio-video feeds for manual analysis, but must automate the entire process by incorporating numerous AI/ML-based technologies including (but not limited to) –

#1. Motion detection, gaze tracking, mouth movement – to identify if a candidate is gesturing/ communicating to anyone else or reading off another device/ screen/ notebook

This feature is heavily reliant on AI/ML for individual candidate’s facial mapping prior to and during the examination. This serves twofold purpose –

1. It must confidently differentiate between a candidate’s submitted photograph/ ID and the individual taking the exam to eliminate impersonation, and
2. It must also rapidly map the candidate’s face vectors and facial landmarks for gaze and mouth movement tracking.

This key component is thus assessed on twin criteria of identification accuracy and mapping speed.

Gaze tracking effectively eliminates blind spots which, if allowed to persist, would render impossible the exclusive dependence on the computer workstation’s inbuilt camera and microphone. These blind spots include any written notes/ mobile phones kept at keyboard level, or the presence of other individuals/ reading material behind the monitor.

Gaze tracking also empowers the AI system to differentiate between a candidate and their still image/ photograph, thereby obviating any attempts at impersonation.

#2. Speech detection and voice sample analysis + Speech-to-text transcription – to identify verbal communication via hidden devices

Much development has already happened in the sphere of speech detection, and several commercial applications are available in the market, the most well-known being Amazon’s Alexa and Google’s Voice search platform.

APIs also exist for speech-to-text transcription and can be integrated with various off-the-shelf applications, be they for virtual meeting platforms or note-making. The text transcript can be automatically cross-checked against the question paper to identify any similarities, thereby indicating verbal cheating.

#3. Visual cognition for facial recognition/ people counter – to detect the presence of unauthorized individuals whether in place of the candidate or assisting the candidate

This correlates with (1) above. AI/ML-based people counters can be trained using GPUs to recognize the presence of persons even when camouflaged or partially visible. They can also flag when no individual is detected in the frame, i.e., when the candidate has left the test workstation/ laptop.

#4. Handwritten text recognition – for rough work analysis

An AI/ML-enabled proctorial system incorporating the aforementioned functionalities can be developed using commonly available tools such as PyTorch (opensource Python library for dynamic execution), DLib, OpenCV, Tensorflow (deep learning framework widely adopted by industries and enterprises) and Natural Language Tool Kit (NLTK).

Post-development training for automation as well as scaling up can be expedited using Public Cloud GPUs so that the system can scale up/ down as required. Integrating with Cloud GPUs has the additional advantage of facilitating real-time storage and logging of the massive quantities of audio-visual and transcript data (running into several TBs!) generated by the system.

Any discrepancy in candidate’s body movements (including gaze), audio-video feed, presence of unauthorized individuals is immediately flagged and time-stamped by the system and communicated either to a human proctor invigilating several candidates in real-time (academic examination scenario), or logged for post-examination sifting of candidates (employment-related examination scenario).

The Way Forward? Rise of the Machine (Invigilators)?

An e-Proctorial system, amalgamating AI-automated invigilation and post-examination manual intervention (if necessitated), coupled with secure browser that flags unauthorized system navigation, can prove to be a gamechanger for undertaking continuous academic and employment-specific assessments.

It can, in fact, go further than manual invigilation since AI-based monitoring is not subject to human failings such as fatigue, disinterest and distraction, nor is it limited by the number of candidates it can realistically proctor simultaneously. In Harry Potter and the Order of the Phoenix, Dolores Umbridge might have been caught off-guard by the delinquent Weasley twins’ mind-boggling exploits, but an AI-enabled proctor would not even bat an unperturbed eyelid in the face of spectacular firecrackers and terrifying dragons spitting searing fireballs!

A more realistic vision of such a software suite involves eventually incorporating it in the larger post-Covid examination scheme as a precursor for preliminary selection, followed by successive rounds of technical assessments and face-to-face interviews.

Several commercial ventures such as Proctoredu, Proctortrack, Mettl and Comprobo have already developed and field -deployed such secure examination software for academic evaluation and employee recruitment. However, successful deployment requires a vision and a strategy.

Ace Cloud Hosting specializes in filling this void. Our experts can assist you in building the next such software suite. Ace Cloud Hosting’s Nvidia A100 GPUs can accelerate AI/ML workloads involved in developing and training such automated proctorial systems by manifolds. These are the same GPUs with which Poland recently constructed its fastest supercomputer ever. Book an appointment with our consultant to know more about NVIDIA A100 GPUs.

You may also like:

• GPU vs. CPU for Image Processing: Which One is Better?
• GPU vs CPU for Data Analytics
• Cloud GPUs: The Cornerstone of Modern AI

Related posts:

How To Find The Best GPU For Deep Learning [2023] Why GPUs for Deep Learning? A Complete Explanation How Many GPUs Should Your Deep Learning Workstation Have? Financial Fraud Detection with Deep Learning and AI