{"doi":"10.1109/trpms.2024.3397318","title":"Context-Aware Transformer GAN for Direct Generation of Attenuation and Scatter Corrected PET Data","abstract":"We present a context-aware generative deep learning framework to produce photon attenuation and scatter corrected (ASC) positron emission tomography (PET) images directly from nonattenuation and nonscatter corrected (NASC) images. We trained conditional generative adversarial networks (cGANs) on either single-modality (NASC) or multimodality (NASC+MRI) input data to map NASC images to pixel-wise continuously valued ASC PET images. We designed and evaluated four cGAN models including Pix2Pix, attention-guided cGAN (AG-Pix2Pix), vision transformer cGAN (ViT-GAN), and shifted window transformer cGAN (Swin-GAN). Retrospective 18F-fluorodeoxyglucose (18F-FDG) full-body PET images from 33 subjects were collected and analyzed. Notably, as a particular strength of this work, each patient in the study underwent both a PET/CT scan and a multisequence PET/MRI scan on the same day giving us a gold standard from the former as we investigate ASC for the latter. Quantitative analysis, evaluating image quality using peak signal-to-noise ratio (PSNR), multiscale structural similarity index (MS-SSIM), normalized mean-squared error (NRMSE), and mean absolute error (MAE) metrics, showed no significant impact of input type on PSNR (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.95$ </tex-math></inline-formula>), MS-SSIM (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.083$ </tex-math></inline-formula>), NRMSE (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.72$ </tex-math></inline-formula>), or MAE (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.70$ </tex-math></inline-formula>). For multimodal input data, Swin-GAN outperformed Pix2Pix (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.023$ </tex-math></inline-formula>) and AG-Pix2Pix (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p \\lt 0.001$ </tex-math></inline-formula>), but not ViT-GAN (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.154$ </tex-math></inline-formula>) in PSNR. Swin-GAN achieved significantly higher MS-SSIM than ViT-GAN (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.007$ </tex-math></inline-formula>) and AG-Pix2Pix (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.002$ </tex-math></inline-formula>). Multimodal Swin-GAN demonstrated reduced NRMSE and MAE compared to ViT-GAN (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p=0.023$ </tex-math></inline-formula> and 0.031, respectively) and AG-Pix2Pix (both <inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p \\lt 0.001$ </tex-math></inline-formula>), with marginal improvement over Pix2Pix (<inline-formula xmlns:mml=\"http://www.w3.org/1998/Math/MathML\" xmlns:xlink=\"http://www.w3.org/1999/xlink\"> <tex-math notation=\"LaTeX\">$p \\lt 0.064$ </tex-math></inline-formula>). The cGAN models, in particular Swin-GAN, consistently generated reliable and accurate ASC PET images, whether using multimodal or single-modal input data. The findings indicate that this methodology can be used to generate ASC data from standalone PET scanners or integrated PET/MRI systems, without re","journal":"IEEE Transactions on Radiation and Plasma Medical Sciences","year":2024,"id":470972,"datarank":0.0,"base_score":0.0,"endowment":0.0,"self_citation_contribution":0.0,"citation_network_contribution":0.0,"self_endowment_contribution":0.0,"citer_contribution":0.0,"corpus_percentile":null,"corpus_rank":null,"citation_count":2,"citer_count":0,"citers_with_citation_signal":0,"citers_with_endowment":0,"datacite_reuse_total":0,"is_dataset":false,"is_dataset_confidence":0.9321,"is_data_producer":false,"deposit_databanks":null,"is_oa":true,"file_count":0,"downloads":0,"has_version_chain":false,"published_date":"2024-01-01","fair_score":null,"fair_percentile":null,"algorithm_id":"datarank_citation_only_1hop_v6","ranking_scope":"data_only","authors":[{"id":490893,"name":"Emily Anaya","orcid":"0000-0003-4171-0247","position":1,"is_corresponding":false},{"id":1237886,"name":"Garry Chinn","orcid":"0000-0002-1467-3228","position":2,"is_corresponding":false},{"id":304556,"name":"Jarrett Rosenberg","orcid":"0000-0003-2706-3290","position":3,"is_corresponding":false},{"id":1144885,"name":"Tie Liang","orcid":null,"position":4,"is_corresponding":false},{"id":490894,"name":"Craig S. Levin","orcid":"0000-0002-4575-5074","position":5,"is_corresponding":false},{"id":1305617,"name":"Mojtaba Jafaritadi","orcid":"0000-0002-4085-4057","position":0,"is_corresponding":true}],"reference_count":70,"raw_metadata":null,"created_at":"2026-07-19T02:05:44.736405Z","pmid":"41809058","pmcid":null,"fwci":null,"citation_percentile":null,"influential_citations":0,"oa_status":null,"license":null,"views":0,"total_file_size_bytes":0,"version_count":0,"fair_f":null,"fair_a":null,"fair_i":null,"fair_r":null,"fair_zscore":null,"fair_rationale":null,"fair_model":null,"fair_agent_version":null,"fair_fulltext_source":null,"fair_has_llm":null,"fair_computed_at":null,"clinical_trials":[],"software_tools":[],"db_accessions":[],"linked_datasets":[],"topics":[]}