Student Privacy Matters: Education Nonprofits, FERPA, and AI

Many AI vendors train their models using customer data to improve accuracy and performance. For education nonprofits, this creates a FERPA violation if student education records become part of a vendor's training dataset.

The risk: Student information from your tutoring program could be used to improve the vendor's AI model, then potentially exposed when that model serves other customers. Even if names are removed, the AI might learn patterns specific to your students that could be revealed through clever prompting or analysis.

What to verify: Contracts must explicitly prohibit vendors from using student data to train AI models, improve algorithms, or enhance services for other customers. This prohibition should survive contract termination, student data can never be retroactively included in training sets.

FERPA allows disclosure of de-identified data, but AI makes true de-identification extraordinarily difficult. FERPA's de-identification standard requires that a "reasonable person" couldn't identify students from the data, even with other available information.

AI systems can often re-identify individuals from supposedly anonymous datasets by cross-referencing multiple data points, recognizing patterns, or combining information from different sources. Research has repeatedly shown that seemingly anonymous data becomes identifiable when analyzed algorithmically.

What this means: Education nonprofits cannot assume that removing names and student IDs creates truly de-identified data when AI is involved. Additional protections, such as data aggregation, statistical noise, or differential privacy techniques, may be necessary to meet FERPA's de-identification standard.

When education nonprofits use AI tools from vendors, they're often sharing student data with third parties. FERPA allows this only under specific conditions, primarily when the vendor qualifies as a "school official" performing services the institution would otherwise handle itself.

To meet the school official exception, vendors must be under the direct control of the educational institution or nonprofit, use student data only for authorized purposes, and not re-disclose the information without permission. Cloud-based AI services complicate this relationship because data may be processed on servers around the world, potentially violating FERPA's requirement that you maintain control.

Common risk: "Free" AI tools that monetize by analyzing user data or selling insights to other customers almost certainly violate FERPA when used with student education records. Education nonprofits must carefully evaluate the business model of any AI tool they consider.

FERPA gives parents and eligible students the right to inspect and review education records, but how does this work when an AI algorithm generates predictions, recommendations, or decisions based on student data?

If an AI system predicts which students are at risk of dropping out, recommends specific interventions, or influences decisions about program placement, parents have a right to understand how those decisions were made. Many AI systems, particularly complex neural networks, operate as "black boxes" where even the vendor cannot fully explain specific outputs.

Compliance requirement: Education nonprofits using AI for decisions affecting students must be able to provide meaningful explanations of how the AI reached its conclusions. This may require requesting "model cards", algorithmic impact assessments, or explainable AI documentation from vendors.

Collect and share only the minimum student data necessary for the AI tool to function. More data means more risk, both of unauthorized disclosure and of the AI system revealing sensitive information through its outputs.

Before implementing an AI tool, map exactly what data the system requires versus what it requests. Many AI systems ask for broad data access when they actually need only specific fields. For instance, a tutoring AI might function perfectly with student age and subject performance data without needing names, addresses, or social security numbers.

Implementation tip: Create data field mappings that show which student information elements each AI system receives. Review these mappings quarterly to identify opportunities to reduce data sharing as you learn how the system actually functions.

FERPA requires that access to student education records be limited to those with legitimate educational interests. When implementing AI tools, this means carefully configuring who can access the system, what data they can view, and which AI features they can use.

Role-based access controls should reflect actual job responsibilities. A volunteer tutor might need AI tools to help with specific subjects but shouldn't have access to student behavioral records or family information. An intake coordinator might need AI-assisted case management but shouldn't see academic assessment data.

Common mistake: Granting broad "administrator" access to multiple staff members because it's easier than configuring granular permissions. This convenience creates significant FERPA compliance risks and should be avoided.

Technology controls provide only partial protection, staff members must understand FERPA requirements and how they apply to AI tools in their daily work. Training should be specific, practical, and reinforced regularly.

Effective training covers what student information is protected under FERPA, which AI tools are approved for use with student data, what constitutes unauthorized disclosure (including showing AI outputs containing student information to unauthorized individuals), and how to respond if they suspect a privacy violation has occurred.

Best practice: Provide scenario-based training that uses realistic examples from your organization's work. "Can I use ChatGPT to help draft an email about a student?" "Can I show the AI-generated progress report to a volunteer mentor?" Staff need answers to these specific questions.

Regular monitoring helps catch compliance issues before they become violations. This includes reviewing access logs to identify unusual patterns, auditing AI system configurations to ensure they match your documented policies, and testing data deletion procedures to verify student information is actually removed when required.

Conduct periodic compliance reviews where you trace student data through your AI systems: where it enters, how it's processed, where it's stored, who can access it, and when it's deleted. This mapping exercise often reveals gaps between policies and actual practice.

Frequency recommendation: Quarterly reviews for organizations with significant AI tool usage, at minimum annual reviews for those with limited AI implementation. After any system changes, configuration updates, or vendor changes, conduct an immediate review.