Dual-encoder fusion and explicit cross-modal alignment are two common strategies for multimodal PET-CT segmentation, but their relative contributions remain unclear when training duration and model capacity are controlled. We therefore reassess dual-encoder fusion and Optimal Transport (OT) regularization in DpDNet using an extended 200-epoch protocol with convergence monitoring and five-fold cross-validation on 75 AutoPET-derived PET-CT cases. The dual encoder shows no significant Dice advantage over a parameter-matched, approximately compute-matched weight-shared two-stream encoder (\(\Delta\text{Dice} = 0.0076\), \(p = 0.61\)), but achieves higher Dice than a parameter-matched early-fusion encoder (\(\Delta\text{Dice} = 0.0465\), \(p = 0.016\)). Because the latter requires \(1.24\times\) the FLOPs, this difference cannot be attributed uniquely to fusion position. OT yields no significant late-training Dice improvement on either the dual encoder (\(\Delta\text{Dice} = 0.0091\), \(p = 0.58\)) or the weight-shared two-stream encoder (\(\Delta\text{Dice} = 0.0080\), \(p = 0.45\)), and a 20-epoch pilot advantage does not persist under the 200-epoch protocol. The previously observed pre-training–OT interaction is likewise not reproduced and remains exploratory. Voxel Dice does not distinguish the matched-capacity configurations, whereas lesion-level detection shows only nominal differences that do not survive multiple-comparison correction. These results indicate that conclusions about multimodal PET-CT segmentation depend strongly on control design, training duration, and evaluation metric.
- Open Access
- Article
Revisiting Dual-Encoder Fusion and Optimal Transport Regularization in PET-CT Tumor Segmentation: A Matched-Control Study under Extended Training
Author Information
Received: 30 June 2026 | Revised: 06 October 2026 | Accepted: 15 October 2026 | Published: 30 October 2026
Abstract
Keywords
PET-CT segmentation | multimodal fusion | dual-encoder architecture | optimal transport | convergence analysis | lesion-level detection
References
- 1.
Gatidis, S.; Früh, M.; Fabritius, M.P.; et al. Results from the autoPET Challenge on Fully Automated Lesion Segmentation in Oncologic PET/CT Imaging. Nat. Mach. Intell. 2024, 6, 1396–1405. https://doi.org/10.1038/s42256-024-00912-9.
- 2.
Litjens, G.; Kooi, T.; Bejnordi, B.E.; et al. A Survey on Deep Learning in Medical Image Analysis. Med. Image Anal. 2017, 42, 60–88. https://doi.org/10.1016/j.media.2017.07.005.
- 3.
Dolz, J.; Gopinath, K.; Yuan, J.; et al. HyperDense-Net: A Hyper-Densely Connected CNN for Multi-Modal Image Segmentation. IEEE Trans. Med. Imaging 2019, 38, 1116–1126. https://doi.org/10.1109/tmi.2018.2878669.
- 4.
Kamnitsas, K.; Ledig, C.; Newcombe, V.F.J.; et al. Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation. Med. Image Anal. 2017, 36, 61–78. https://doi.org/10.1016/j.media.2016.10.004.
- 5.
Chartsias, A.; Joyce, T.; Giuffrida, M.V.; et al. Multimodal MR Synthesis via Modality-Invariant Latent Representation. IEEE Trans. Med. Imaging 2018, 37, 803–814. https://doi.org/10.1109/tmi.2017.2764326.
- 6.
Fu, X.; Bi, L.; Kumar, A.; et al. Multimodal Spatial Attention Module for Targeting Multimodal PET-CT Lung Tumor Segmentation. IEEE J. Biomed. Health Inform. 2021, 25, 3507–3516. https://doi.org/10.1109/jbhi.2021.3059453.
- 7.
Chen, T.; Kornblith, S.; Norouzi, M.; et al. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of the 37th International Conference on Machine Learning, Online, 13–18 July 2020; pp. 1597–1607.
- 8.
Courty, N.; Flamary, R.; Tuia, D.; et al. Optimal Transport for Domain Adaptation. IEEE Trans. Pattern Anal. Mach. Intell. 2017, 39, 1853–1865. https://doi.org/10.1109/tpami.2016.2615921.
- 9.
Huang, J.; Chen, S.; Liang, X.; et al. C2MAOT: Cross-Modal Complementary Masked Autoencoder with Optimal Transport for Cancer Segmentation in PET-CT Images. In Medical Image Computing and Computer Assisted Intervention—MICCAI 2025, Proceedings of the 28th International Conference, Daejeon, South Korea, 23–27 September 2025; Springer: Cham, Switzerland, 2025; Volume 15960, pp. 87–97. https://doi.org/10.1007/978-3-032-04927-8_9.
- 10.
Gatidis, S.; Kuestner, T. A Whole-Body FDG-PET/CT Dataset with Manually Annotated Tumor Lesions (FDG-PET-CT-Lesions). Available online: https://www.cancerimagingarchive.net/collection/fdg-pet-ct-lesions/ (accessed on 5 October 2026).
- 11.
Liang, X.; Huang, J.; Han, L.; et al. DpDNet: An Dual-Prompt-Driven Network for Universal PET-CT Segmentation. In Medical Image Computing and Computer Assisted Intervention—MICCAI 2025, Proceedings of the 28th International Conference, Daejeon, South Korea, 23–27 September 2025; Springer: Cham, Switzerland, 2025; pp. 163–172. https://doi.org/10.1007/978-3-032-04978-0_16.
- 12.
Neha, F.; Shukla, D.K. Radiomics in Medical Imaging: Methods, Applications, and Challenges. J. Imaging 2026, 12, 220. https://doi.org/10.3390/jimaging12060220.
- 13.
Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Proceedings of the 18th International Conference, Munich, Germany, 5–9 October 2015; Springer: Cham, Switzerland, 2015; pp. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28.
- 14.
Çiçek, Ö.; Abdulkadir, A.; Lienkamp, S.S.; et al. 3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2016, Proceedings of the 19th International Conference, Athens, Greece, 17–21 October 2016; Springer: Cham, Switzerland, 2016; pp. 424–432. https://doi.org/10.1007/978-3-319-46723-8_49.
- 15.
Milletari, F.; Navab, N.; Ahmadi, S.A. V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation. In Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA, 25–28 October 2016; pp. 565–571. https://doi.org/10.1109/3dv.2016.79.
- 16.
Neha, F.; Bhati, D.; Shukla, D.K.; et al. An Analytics-Driven Review of U-Net for Medical Image Segmentation. Healthc. Anal. 2025, 8, 100416. https://doi.org/10.1016/j.health.2025.100416.
- 17.
Hatamizadeh, A.; Tang, Y.; Nath, V.; et al. UNETR: Transformers for 3D Medical Image Segmentation. In Proceedings of the 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3–8 January 2022; pp. 1748–1758. https://doi.org/10.1109/wacv51458.2022.00181.
- 18.
Cao, H.; Wang, Y.; Chen, J.; et al. Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation. In Proceedings of the Computer Vision—ECCV 2022, Tel Aviv, Israel, 23–27 October 2022; pp. 205–218. https://doi.org/10.1007/978-3-031-25066-8_9.
- 19.
Chen, J.; Lu, Y.; Yu, Q.; et al. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv 2021, arXiv:2102.04306.
- 20.
Ganin, Y.; Ustinova, E.; Ajakan, H.; et al. Domain-Adversarial Training of Neural Networks. J. Mach. Learn. Res. 2016, 17, 1–35.
- 21.
Tzeng, E.; Hoffman, J.; Saenko, K.; et al. Adversarial Discriminative Domain Adaptation. In Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, 21–26 July 2017; pp. 2962–2971. https://doi.org/10.1109/cvpr.2017.316.
- 22.
Arjovsky, M.; Chintala, S.; Bottou, L. Wasserstein GAN. arXiv 2017, arXiv:1701.07875.
- 23.
Gulrajani, I.; Ahmed, F.; Arjovsky, M.; et al. Improved Training of Wasserstein GANs. In Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4–9 December 2017; Volume 30, pp. 5767–5777.
- 24.
Peyré, G.; Cuturi, M. Computational Optimal Transport with Applications to Data Sciences. Found. Trends Mach. Learn. 2019, 11, 355–607. https://doi.org/10.1561/2200000073.
- 25.
Frogner, C.; Zhang, C.; Mobahi, H.; et al. Learning with a Wasserstein Loss. In Proceedings of the 29th International Conference on Neural Information Processing Systems, Montreal, QC, Canada, 7–12 December 2015; Volume 28, pp. 2053–2061.
- 26.
He, K.; Chen, X.; Xie, S.; et al. Masked Autoencoders Are Scalable Vision Learners. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 15979–15988. https://doi.org/10.1109/cvpr52688.2022.01553.
- 27.
Zhou, Z.; Sodha, V.; Pang, J.; et al. Models Genesis. Med. Image Anal. 2021, 67, 101840. https://doi.org/10.1016/j.media.2020.101840.
- 28.
Azizi, S.; Culp, L.; Freyberg, J.; et al. Robust and Data-Efficient Generalization of Self-Supervised Machine Learning for Diagnostic Imaging. Nat. Biomed. Eng. 2023, 7, 756–779. https://doi.org/10.1038/s41551-023-01049-7.
- 29.
Tang, Y.; Yang, D.; Li, W.; et al. Self-Supervised Pre-Training of Swin Transformers for 3D Medical Image Analysis. In Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA, 18–24 June 2022; pp. 20698–20708. https://doi.org/10.1109/cvpr52688.2022.02007.
- 30.
Isensee, F.; Jaeger, P.F.; Kohl, S.A.A.; et al. nnU-Net: A Self-Configuring Method for Deep Learning-Based Biomedical Image Segmentation. Nat. Methods 2021, 18, 203–211. https://doi.org/10.1038/s41592-020-01008-z.
- 31.
Cuturi, M. Sinkhorn Distances: Lightspeed Computation of Optimal Transport. In Proceedings of the 27th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 5–10 December 2013; Volume 26, pp. 2292–2300.
- 32.
Andrearczyk, V.; Oreiller, V.; Abdelaal, M.A.; et al. Overview of the HECKTOR Challenge at MICCAI 2022: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET/CT. In Head and Neck Tumor Segmentation and Outcome Prediction, Proceedings of the 3D Head and Neck Tumor Segmentation in PET/CT Challenge, Singapore, 22 September 2022; Springer: Cham, Switzerland, 2023; pp. 1–30. https://doi.org/10.1007/978-3-031-27420-6_1.
This work is licensed under a Creative Commons Attribution 4.0 International License.
