Medical image de-identification is essential for data sharing, but conventional OCR-based redaction can expose protected health information (PHI) to the service that performs redaction. We present a client-cloud framework that applies fully homomorphic encryption (FHE) to privacy-preserving medical image de-identification. The client localizes text, segments character patches, encrypts normalized inputs with CKKS, and performs final correction and masking, while the cloud evaluates an FHE-compatible VGG-style character recognizer and CharCNN PHI classifier over ciphertexts. The system reduces encrypted inference cost through polynomial activations, average pooling, diagonal encoding, Toeplitz-style convolution lowering, and BSGS rotation reuse. Prototype results show 100% plaintext-ciphertext prediction consistency, 83.15% corrected character accuracy, and 97.1% encrypted PHI-classification accuracy, with bootstrapped nonlinear layers remaining the main bottleneck.



