Situational Analysis

During strategic planning sessions, Sunstar Paramedics identified the need to streamline the pre-hire process, improve the quality of training of the new EMTs and Paramedics, and provide the Field Training Officers (FTO) with more leadership skills and abilities.

Goals

• Implement a new pre-hire online assessment test to improve the efficiency and quality of the orientation and training process
• Decrease the number of substantiated Quality Assurance Reviews (QAR).
• Develop a formal training academy for provisional EMTs and Paramedics and reduce clearance time to the field.
• Increase engagement and provide professional development for the Field Training Officer program (increase instructor ratings for FTOs).

Planning & Implementation

Timeline

Pre-hire online assessment exam:

September 2015 – Developed the idea to eliminate the practical scenario and replace with online testing scenarios. The clinical services team performed analysis to determine the main qualities desired from the practical test. Decision making, confidence, and comfort with the material were determined to be the main attributes being tested by the scenario and psychomotor competency could be trained and tested after hiring. The test was designed as a virtual scenario which prohibited testers from going back, and also timed each question to elicit the desired qualities. Each question was categorized as basic, intermediate or advanced, and local/regional questions were also designated. This allowed out-of-area applicants to be tested fairly. If out-of-area applicants did poorly on local questions but well on others, they were deemed as trainable in theory, and brought in for an interview.

October 2015 -The clinical services team worked with third party, ReviewNet, to beta test a new software allowing companies to write their own test questions. At the time, ReviewNet was strictly an IT testing company. During this time a paper version of the test was used to begin validation of the material, while engineers developed the online version.

February 2016 – The first online tests go live, with success. Many EMT students performed worse than expected, while medics scored better than expected. A manual review was performed of each failing EMT grade.

August 2016 – EMT scoring pattern was identified in successful candidates and passing criteria was adjusted to account for overall score, section scores, overall time taken, and time per question.

January 2017 – EMT test version 2 is created based on employee feedback, new protocols, implementation of BLS transport, and statistical analysis. New scoring standard was set for EMTs. Medic test deemed valid and continued.

Restructured training focus:

October 2015 – The fiscal year average had a Paramedic being cleared in 100 days or more and EMTs over 90 days with a first time Capstone (final clearing test) pass rate at 46%. This cost a significant amount of labor hours, and was an unsustainable cycle for meeting the needs of our operation. The strategic planning group developed a goal of 75 days for paramedics and 50 days for EMTs to be County certified and cleared to the system.

January 2016 -The clinical services team developed a pipeline academy for each of the disciplines that fit within this timeline. In addition to meeting the time standard the clinical services department developed a curriculum that increased the Pinellas County capstone pass rate for provisional employees from 46% to 95% and shortened the clearing time to 70 days for paramedics and 50 days for EMTs.

March 2016 through YTD – The clinical services team developed and implemented a new style of classroom training in conjunction to the field shift training. New “phase evaluation” days made use of our medium fidelity simulation mannequins,credentialed FTO instructors, and clinical services coordinators. The classroom days consisted of mini monitor, airway, and assessment stations that eventually built into a full call scenario which put the Paramedics and EMTs through arrival, assessment, transport, and arrival to destination components. The refocusing on drill-like training with a focus on assessments increased the proficiency level of the newly cleared clinicians, lowered the occurrence of quality assurance review (QAR) incidents, and decreased the clearing time of new employees. The areas of opportunity identified to focus on were medication administration, electrotherapy, patient assessment, and crew resource management.

FTO leadership—clinical, professional and educational development:

The final area of opportunity for the onboarding process is the FTO Program. This is a group of field personnel that complete the ‘on the job’ training portion of our provisional academy. This group consists of over 100 EMTs and paramedics that have displayed excellent job performance, and expressed a desire to train new employees. This group lacked an objective measure of performance and feedback, as well as any formal training.

January 2016 – a new format for the FTO group was initiated with a promise of professional, educational, and clinical development. This was accomplished by holding quarterly meetings throughout 2016. The meetings involved guest speakers (neurologist/trauma surgeons), cardiac/airway labs using pig hearts and lungs, and professional development by offering free instructor rating classes (ACLS, BLS, AMLS, etc.)

January of 2017 – The clinical services department developed quarterly training and professional development sessions, and a FTO Report Card that measures key job functions. The report card is given to the group monthly, and any individuals identified to be under the standard are placed on performance improvement plans to help correct the deficiency.

Results

We achieved our goals with a reduction in clearance times for EMTs and Paramedics, decreased labor cost and time for the pre-hire clinical assessment, decreased the amount of quality assurance reviews and increased the number of instructor ratings for the FTO group.

Impact

The biggest advantage of the hiring testing process was removing a barrier for employment based on test taking ability. For the old process, candidates were surprised with an extensive written test and practical scenario, often in a suit and tie and already anxious. Many candidates chose not to retest if they were not successful and simply moved on.

While fewer candidates fail the new test, those that do immediately end the hiring process. This saves several hours of the recruiter’s time, as well as the clinical coordinators who used to administer the exam. There was only a minor increase in average per month applicants, even though there was a large drop-off in clinical failures. This was partly due to candidates washing out of the process at other points in the process (often the interview or agility test). Therefore the change streamlined the process saving time, labor cost, and inappropriate hires, without adversely affecting quality.

The new streamlined training program did indeed reduce the amount of overhead and expense by using training time more responsibly and eliminating training that did not meet core objectives.

There was a significant drop in substantiated quality assurance reviews (QARs), which contain performance remediation plans for clinical issues. It can be reasoned that the increased focus on protocol training and higher quality of field training officer program contributed to this decrease. This is validation that our practices did indeed save time and money while not sacrificing quality of patient care – in fact patient care quality improved under the new program!

Our efforts received recognition in the August 2017 JEMS in an article outlining the improvement process as well as recognition by receiving the State of Florida’s “EMS Educator of the Year” Award for 2017.

Sunstar is currently tracking new outcome measures like patient engagement and documentation quality, which will likely show similar positive trends in the future.

(From JEMS): Restructuring the clinical services department hasn’t just improved internal communications and processes; it’s also enhanced Sunstar Paramedics’ relationships with local agencies. Seven different agencies within Pinellas County have reached out to Sunstar Paramedics and arranged for their employees to go through the redesigned new hire training. This training partnership forges better interagency relationships, allows Sunstar Paramedics crews the opportunity to train with local fire departments and other neighboring EMS agencies, and has led to marked improvements in the EMS culture in Pinellas County.

Budget

Since this employee related project was initiated to improve efficiency, lower cost, and current resources were used a budget was not necessary. The project positively impacted our budget by reducing cost of new hire orientation and training by half.

Files: Amby 2017 Sunstar Employee Training Program FINAL.pdf