Google has released HEIR (Homomorphic Encryption Intermediate Representation), an open-source compiler and development toolchain that allows pre-trained AI models built for conventional, unencrypted inputs to operate on encrypted data instead. The company announced the platform in a recent publication, positioning it as a solution to privacy and security challenges that arise when cloud services need to process sensitive user information or when proprietary AI models are deployed on user devices.
The toolchain enables developers to write programs in Python, annotate which data types should be encrypted, and compile the code for homomorphic execution. Google has applied HEIR to several use cases: enabling private content recommendations without exposing user data, detecting credit card fraud while protecting sensitive financial information, identifying network intrusions without revealing packet contents to the service provider, and recognizing hotwords in audio streams without disclosing the underlying recordings. The platform uses an intermediate representation as an abstraction layer to handle complex models across different dialects, though the process isn't yet automated and requires multiple steps, starting with using torch_mlir to export PyTorch models to MLIR.
Performance concerns dominated early community response. One Hacker News reader noted that homomorphic encryption and similar techniques carry "very high overheads(~10^3) on inference tasks," while another provided specific benchmarks: a 64-bit equality operation takes 80 milliseconds, addition and subtraction require 100 milliseconds, and division can take as long as 8 seconds. However, according to the report, the outlook for large language models may be more favorable since their computations rely heavily on addition and multiplication, while branching—among the operations that fully homomorphic encryption handles most poorly—isn't entirely relevant to LLM architecture. One commenter suggested that "1000x overhead would make some image classification tasks go from 1ms to 1s. That's viable for some applications."
The privacy trade-off emerged as a central theme in discussions about the technology. Homomorphic encryption fundamentally changes the calculation between convenience and security by allowing computations to be performed directly on encrypted data—servers can process ciphertexts and return encrypted results without exposing any underlying information. This addresses challenges with end-to-end encryption, including enabling services such as spam and virus detection to operate on encrypted user data and reducing the risk of exposing proprietary AI models when they're deployed on users' devices. Yet some observers argued that local deployment on personal hardware remains the most private option, with one noting that "when you want to keep a query/data private it's maybe worth the extra $ KW" required for running models locally. While Google included benchmarking code in HEIR's repository, it hasn't provided any figures about the platform's relative speed when applied to large language models. The company describes homomorphic encryption as "a rapidly maturing technology" that could enable new categories of privacy-preserving services, though real-world performance at scale remains to be demonstrated. Whether organizations will accept the computational overhead in exchange for processing sensitive data in the cloud—or instead choose local deployment—will likely depend on how well the technology scales beyond the use cases Google has highlighted. The calculus may shift as encryption techniques improve and cloud infrastructure becomes more optimized for homomorphic workloads.