Career and Technical Education (CTE) Program Performance Reporting and Audit Review Rulemaking Workshop #1 – Frequently Asked Questions
BACKGROUND
In House Bill 1507 (2021), Florida Department of Education (FDOE) was granted rulemaking authority to amend Rule 6A-10.0342, Florida Administrative Code (F.A.C.), Career and Technical Education Program Performance Reporting, commonly known as the CTE audit rule. This rulemaking is required by section 1003.491(5), Florida Statutes (F.S.).
Note: The FDOE has postponed drafting and publicly sharing a proposed version of the rule for public comment because “related matters are not sufficiently resolved to enable the FDOE to address a statement by rulemaking,” per s. 120.54, F.S. As identified in s. 1003.491(5), F.S., related matters include the “alignment to the Framework of Quality,” under s. 445.004(4), F.S., and the “alignment of offerings at the K-12 and postsecondary levels with credentials or degree programs identified on the Master Credentials List,” under s. 445.004(4), F.S. All agencies will be notified when a date is established for review and feedback of the draft rule language.
The CTE audit rule cannot be fully developed with fidelity until the following other projects have been completed in fall 2022 and early 2023:
•	The Labor Market Estimating Conference (LMEC) analyses of occupational earnings;
•	The LMEC analyses of labor market supply, demand and gaps; and
•	The Credential Review Committee’s (CRC) Framework of Quality.
Currently, the FDOE has not created draft rule language to share with the stakeholders. 
CTE AUDIT 
The goal of the CTE audit is to systematically measure secondary and postsecondary CTE programs for quality and to ensure alignment of existing offerings with employer demand, postsecondary credentials and professional industry certifications.  
For more information and the most recent CTE audit data, please visit Career Pathways.
RESOURCEFUL LINKS
•	The revision to s. 1003.491(5), F.S., as created in House Bill 1507, is located at the following link: The Florida Career and Professional Education Act, s. 1003.491, F.S. 
•	As of April 2022, the CRC is still in development of the Framework of Quality (FoQ) and the Master Credentials List (MCL). 
•	A detailed account of all CRC-mandated requirements per s. 445.004(4), F.S., such as required organization representation, FoQ and MCL should be reviewed. The CRC is expected to adopt the final MCL in January 2023. For the most recent developments on the FoQ, MCL and any additional specific questions regarding the CRC, please visit the CRC homepage. 
•	After the February 17, 2022, convening of the LMEC, the proposed labor market supply and demand data were not adopted by the state. Therefore, the development of the CTE audit rule is also dependent on the completion of the work of the LMEC as defined in s. 216.136, F.S. For the most recent developments on LMEC, please visit the LMEC homepage. 
WORKSHOP RECORDING
As part of the rule development process, the FDOE conducted a rule development workshop to educate the field and discuss proposed changes to the CTE audit process that will come into effect via rule. 
•	Workshop #1 for Rule 6A-10.0342, F.A.C. was held via webinar from 1:00 – 2:00 p.m. ET. on Friday, January 21, 2022. The workshop discussed the intent of the CTE audit rule and possible forthcoming changes for the CTE audit process.
Please visit the CTE audit website at Career Pathways and a copy of the recording and presentation are located in the “Correspondence” section of the website. 
Recording title: 2022-01-21 CTE Audit Rulemaking Workshop Recording.
Presentation: 2022-01-21 CTE Audit Rulemaking Workshop Presentation.
CONTACT INFORMATION
For additional questions concerning the first CTE Program Performance Reporting and Audit Review Rulemaking workshop, please reach out to Keith Richard, Statewide Director of CTE Quality (Keith.Richard@fldoe.org) and Brittany Salaam, Program Analyst (Brittany.Salaam@fldoe.org). Please include Brittany Salaam on all correspondence to ensure that your question is logged.
Program Removal and the CTE Audit Rule
1.	What will cause a program to be “discontinued” in a local district or college, per the CTE audit?
The CTE audit rule will outline which criteria will be used to recommend improvement plans or discontinuation of a locally offered program. The FDOE has not yet developed this process and it will be created during the CTE audit rule development process. 
2.	The revision of s. 1003.491(5), F.S., requires that the FDOE phase out CTE programs in local districts or colleges if the programs do not meet performance metric requirements with the new CTE audit rules. Will the FDOE establish a probationary period and process for underperforming programs?
Yes. The intent is to develop a rule that will establish a process for dialogue between the FDOE and districts or colleges with underperforming programs. Programs flagged as underperforming, and then confirmed to be underperforming by the local level agency, will then go into a probationary period to track for program performance improvement. The intent is to not eliminate programs via the CTE audit rule simply because the program is underperforming at one point in time. Rather, the intent of the CTE audit rule is to develop a process to work with local level agencies to create specific improvement plans for programs that are underperforming on specific performance measures.
If a district or college agrees that the program is underperforming but does not wish to develop a program improvement plan, the FDOE will recommend this program for removal. If a district does develop an improvement plan, the FDOE will review the program later to determine if the program is showing improvement. 
3.	If the FDOE recommends a program for removal, how long will the district or institution have to phase out the program?
We are currently determining the rule process and have yet to establish a timeframe for phasing out a program. In our rule development process we will ensure that any programs that the FDOE recommends for removal will have a sufficient teach out period. The CTE audit rule will provide guidelines for proper teach out procedures. The teach out procedures will ensure all currently enrolled CTE students have the opportunity to complete their credentials or transfer into a related program. 
4.	If a District Technical College and a State College both offer a program, how will it be determined which institution will be required to remove their program? 
The FDOE is currently investigating how we will address this section of the law to ensure that we are meeting the educational needs of local communities. We do not intend to recommend the removal of programs that are sufficiently serving different local communities. 
5.	In previous years, CTE programs were allowed to demonstrate market demand with letters of program support from local employers. Will districts and institutions be given the opportunity to “defend” a program set for removal (based on not meeting market demand criteria)?  
Yes. Currently, in the “second phase” of CTE audit analysis for local market demand, the FDOE evaluates programs using the district or college Comprehensive Local Needs Assessment (CLNA) Labor Market Alignment (LMA) submitted to the FDOE in compliance of Perkins V. The agency does not intend to move away from this standard with the CTE audit rule. Further, the FDOE will notify districts or colleges if their programs that are flagged as not meeting market demand. When this occurs, districts or colleges will have the opportunity to provide additional evidence of market demand.
6.	The removal of programs with regional duplication is a new requirement in the statute. Can you explain further the meaning of regional duplication and the removal process? 
The agency is in the process of understanding the meaning for several of the new statutory requirements. The statute does not define regional duplication. 
If your district or institution would like to provide additional feedback concerning unwarranted duplication or any of the CTE audit rule metrics, please feel free to reach out to the Office of CTE Quality (direct contact information is in the contact information of this document).
Please see question #3 for additional details. 
7.	Many districts and institutions provide new programs based on the evolving market demand. Is the FDOE preparing for the addition of new program approvals once the rule is completed?  
Yes, the FDOE will be working on the details for a new program approvals process in accordance with the provisions in section 35 of House Bill 1507 (s. 1011.80(2), F.S.). It should be noted that the FDOE does not intend for the approval process to serve as a barrier to starting new programs that are in demand.
CTE Audit Rulemaking and Related Florida Initiatives
8.	There is potential overlap in credential (industry) certifications. If the MCL will include those eligible for Performance Based Incentive Funding (PBIF), will certifications categorized as "Perkins" but not PBIF be included on the list? 
It is possible that some credentials that are currently only approved for Perkins will be reviewed and approved as part of this process. This particular question would best be answered by the CRC. The CRC responsibilities include, in addition to degrees and certificates, identifying all non-degree credentials (industry certifications) that meet the FoQ criteria to appear on the MCL. However, questions regarding the CRC and the MCL should be directed to the CRC in the link provided the resourceful links section of this FAQ. 
9.	How will the CRC performance incentives alter the CTE audit?  
The Credential Review Committee’s (CRC) work is still in development. Please see the CRC website for additional details. Section 1003.491(5), F.S., indicates that the CTE audit must examine, “alignment of offerings with the framework of quality under s. 445.004(4), F.S. alignment of offerings at the K-12 and postsecondary levels with credentials or degree programs identified on the Master Credentials List under s. 445.004(4)” 
The FoQ referenced in the statute s. 1003.491(5), F.S., refers to the evaluation criteria currently still in development by the CRC, and is specific to wage thresholds, labor market demand thresholds and “stackability criteria” for non-degree credentials. The MCL referenced in the statute s. 1003.491(5), F.S., refers to the list of programs that have met the FoQ and approved by the CRC, which is also still in development by the CRC.
The FDOE is still investigating how the intent and goals of the CRC’s FoQ and MCL are similar to, or are different from, the intent and goals of the CTE audit process. While we expect there to be some alignment between the two processes, it is not necessarily the intent of the FDOE for the CTE audit process to be identical to the FoQ and the MCL. Whereas the FoQ and MCL are intended to be used to identify the programs most deserving of performance incentive funds, this is not the same intent of the CTE audit.
10.	Since the MCL is set to be aligned to programs, has the CRC stated if career- themed courses (CTCs) will also be aligned to the MCL?
The CTE audit rule does not handle CTCs. For additional information on CTCs, please visit CAPE Secondary Resources. 
CTCs are a course, or a series of courses, that lead to an industry certification on the CAPE Industry Certification Funding List (ICFL). 
11.	When you mention FoQ, are you referring to the Association for Career and Technical Education (ACTE) framework?
No, the FoQ being referenced in the CTE audit rule is currently being developed by the CRC in accordance with s. 445.004(4), F.S.
12.	How does this process differ from the Council of Occupational Education (COE) accreditation criteria? 
Although the CTE audit and COE accreditation measure similar concepts, these metrics are not operationalized in the exact same ways. The current measurements of the CTE audit are not identical to the measurement used in the COE accreditation process.
Please refer to the data definitions included in the CTE audit excel data files or the “CTE Audit Methodology FAQ” to better understand the measurement of metrics in the CTE audit. 
13.	Will performance be aligned to COE accreditation performance indicators including completion, placement, and licensure? 
The COE performance metrics are not currently something that the FDOE maintains, nor is it something that the FDOE can replicate identically using our access to student record-level data. However, during the CTE audit rulemaking process, we may establish that the COE accreditation performance data can be used as local level evidence to either refute or support the conclusions that the FDOE draws (using FDOE data) for the CTE audit.
Please see question #12 for additional details.  
14.	How much duplication of effort or overlap is there in the information requested by the state for the CTE audit and for the Workforce Program Inventory? 
There will be no duplication of effort. The Workforce Program Inventory that is currently in development through a survey to the districts and colleges will be incorporated into the next data pull for the CTE audit. This forthcoming inventory will allow us to better flag (in the CTE audit) programs that are actually being offered at the local level and exclude programs (from local CTE audit review) that are not being offered in a given local context. Currently, we do not have an inventory of which programs are active at the local level. This inventory will allow to filter out programs from the CTE audit data that are not actually being offered locally.
15.	Is there any connection between the CTE audit rule and the Workforce Development Information System? Do you know what the output of the Workforce Development Information System will be used for (for the Technical Centers and State Colleges)?  
The Workforce Development Information System is the name used by the Florida Legislature in House Bill 1507 to describe the need for a data dashboard tool. The FDOE has renamed this project the Education Meets Opportunity Platform (EMOP). The EMOP tool will include the CTE audit data and allow the FDOE agency-level staff to better visualize the data and performance of programs across the state. The first iteration of EMOP tool is intended for agency-level staff and is scheduled to be completed in fall 2022. The FDOE is exploring how a second interaction of the EMOP can make sections of the EMOP (including the visualization of CTE audit data) available to college and district-level stakeholders. 
16.	How will the CTE Audit process mesh with the LMEC?
The LMEC is focused on creating projections of occupational supply, demand and gaps for each occupation in Florida (analyses at the Standard Occupational Classification (SOC) code level). These data are integral to the CRC process. Because the CTE audit is now intended to have some alignment to the CRC process, the future of the CTE audit may include the occupational supply, demand and gap analyses as determined by the LMEC. If this change does occur, it will not be until 2023. 
Currently, the occupational projections included in the CTE audit are generated by the Department of Economic Opportunity (DEO). In short, it is possible that the CTE audit will no longer use the DEO occupational projections and instead incorporate the LMEC occupational projections. This is still to be determined as the LMEC and CRC work are still in progress.