Integrated platform for high-precision, high-throughput in-flow CTC detection
Our platform unites three orthogonal technologies—patented saw-tooth inertial microfluidics, DHM, and dual-channel immunofluorescence (IF)—into an optimized in-flow process that overcomes both the low abundance and phenotypic heterogeneity of CTCs (Fig. 1). To begin, whole blood is diluted in a proprietary buffer media aimed at minimizing cell loss through adhesion to plastic surfaces. Next, it flows through Astrin’s patented saw-tooth inertial microfluidic chip (WO 2024/064911 A1), which enriches CTCs by depleting hematologic background. This process removes over 99.999% of RBCs and approximately 99.6% of WBCs (Fig. 2), while retaining 95% of spiked cancer cells (Supplementary Information Section 1, SI §1). Overall, this results in more than a 100-fold enrichment of CTCs in the processed sample (Fig. 2). A detailed description of the microfluidic chip and sample preparation protocols is provided in Methods and SI §1.
Fig. 1: System overview, depicting the detection of CTC candidates based on a combination of microfluidic inertial enrichment, digital holographic microscopy signature, and immunofluorescence (IF) expression.
Samples begin as whole blood prior to microfluidic enrichment, during which red blood cells are primarily depleted while retaining CTCs and most WBCs. Subsequently, holograms of each cell and IF signal from the field-of-view as a whole are captured during passage through a second microfluidic chip. Detections in the IF signal for emitting cells are visible as broad peaks whose time-scale is inversely proportional to the frame rate (i.e., equivalent to the time taken for a cell to passage the field-of-view). Holograms pass through a neural network to classify each cell and detect CTCs, while the IF data can be used for further filtering operations for cell enumeration.
Fig. 2: Cumulative end-to-end system depletion performance, where each stage builds on the depletion of the prior stage.
Therefore, immunofluorescence depletion numbers are not from immunofluorescence data alone, but rather in conjunction with holography data. Cell counts for whole blood are based on Cleveland Clinic complete blood count data for red blood cells and white blood cells in males8. Microfluidic, holographic, and immunofluorescence depletion data are measured on the imaging system. Fold depletion is calculated in terms of means, rather than medians, as the median value for immunofluorescence stage is zero for RBCs. Each of the four stages shown here mirrors those depicted in Fig. 1.
Subsequently, the enriched sample is hydrodynamically focused into a straight microfluidic channel and analyzed using our dual-modality imaging system (details in Methods). A digital camera captures the holograms generated by a pulsed 405 nm laser, encoding both the optical and morphological information of the cells (Fig. 1). A customized deep learning neural network processes these holograms in real-time (as described in the next section and detailed further in Methods). The network produces a spatial probability heatmap highlighting potential CTC locations, as learned from training on annotated images of cancerous and healthy cells.
Such in-flow CTC classification via holographic imaging provides an avenue for a completely label-free enrichment platform and can improve over time as training datasets grow and deep learning models evolve. Crucially, the label-free DHM system remains compatible with IF-based molecular labeling to enhance the accuracy of the system or provide secondary enumeration metrics (e.g., degree of EpCAM expression). For the system presented herein, two photomultiplier tubes (PMTs) are integrated to capture IF signals emitted by tumor-specific antibody conjugates upon excitation by a co-linear 488 nm laser (Fig. 1). A custom signal processing algorithm (SI §2.1) identifies true cellular fluorescence peaks amid background noise in each PMT channel, performs cross-channel matching, and integrates the resulting IF readouts with predictions from the DHM classifier.
Overall, the integration of biophysical profiling from deep learning based DHM and biochemical readout from IF substantially reduces candidate events and false positives (Fig. 2). The DHM classifier and IF channel each reduce the nucleated cell background by approximately 100 fold, yielding a post fusion false positive rate of about 1 cell per mL, which is sufficient for patient level discrimination in our cohort. At our prespecified operating point, DHM based classification does not meet the per milliliter false positive constraint, so a fused DHM and IF readout is necessary. Conversely, IF by itself lacks adequate sensitivity and stability due to variable antigen expression and occasional nonspecific fluorescence. This dual modality design also mitigates dependence on antigen status, for example during EMT with EpCAM downregulation, because antigen-low CTCs can still be identified by their label-free holographic signatures, while spurious fluorescence is suppressed by the DHM classifier.
Deep Learning Framework
The benefits of using DHM for label-free CTC detection are counterbalanced by several intrinsic challenges. Chief among these is the extreme scarcity of CTCs, which makes it difficult to compile a sufficiently large and diverse set of positive examples for training a robust detection model. Available public datasets are limited and primarily consist of cultured cancer cell lines, and holographic image datasets are even rarer. However, patient-derived CTCs exhibit far broader phenotypic heterogeneity than any single cell line can represent43,44. Therefore, we train the detector to recognize deviation from a large set of white and red blood cells from healthy patient donors, and we regularize the positive class with a diverse set of epithelial cancer cell lines acquired across many independent imaging sessions. This design aims to preserve generalizability to true CTCs, and avoids biasing the label-free model toward antigen-high phenotypes that IF preferentially labels.
Figure 3 summarizes the strategy we adopted to address these challenges while maintaining the real-time throughput required for in-flow assays. We formulated the detection task as an image-to-heatmap transformation in which a streamlined High-Resolution Network (HR-Net)45(SI §3.1) directly yields a probability surface. Local maxima above a chosen confidence threshold correspond to putative cells. Representing cells as Gaussian key points, rather than bounding boxes, obviates the need for precise manual annotation of poorly-defined holographic fringes. The number and depth of model layers were customized to strike a balance between descriptive depth and inference speed—ensuring both high prediction accuracy while supporting live processing of full-field holograms (architectural details in Methods).
Fig. 3: Deep learning model training process.
Positive and negative samples are generated by imaging cell lines and healthy blood (post-microfluidic enrichment) in separate samples. All images derived from healthy blood are labeled negatively (i.e., lacking any Gaussian keypoint label). Cell lines are spiked into buffer, and all nucleated cells are labeled as positive via a pseudo-labeler neural network, which is itself trained from human-generated labels. The pseudo-labeler is never applied to images with blood cells. Although the pseudo-labeling model is trained from human-generated labels, the CTC detection model can be trained in an automated fashion, enabling large datasets. During each training epoch, the effects of false negatives from the pseudo-labeler (i.e., unlabeled cell line cells) are mitigated using pixel-wise asymmetric loss, wherein CTC model predictions lacking a corresponding pseudo-labeler label (depicted as yellow Gaussian blobs) are penalized at 1/10th the rate of false positives and false negatives. Each training epoch also uses hard sample mining such that images from low-performing runs from healthy controls are preferentially sampled in subsequent epochs.
Training data were deliberately assembled to counter both the rarity of CTCs and their morphological diversity. Two complementary streams were employed (Fig. 3, left side). The negative stream comprised enriched healthy blood, exposing the network to the full spectrum of WBCs, RBCs, debris, and background variation. The positive stream consisted of suspensions of five histologically distinct cancer cell lines imaged in buffer. No blood cells were mixed with these cancer cell line runs, and therefore every nucleated object in these frames is a putative cancer cell. Because such images lack pixel-level labels, a companion HR-Net was first trained on select hand-annotated frames to create provisional “pseudo-labels”. It is important to note that pseudo-labels are only applied to the cell lines. The pseudo-labeler is never given the more difficult task of classifying between healthy and cancer, only to label all nucleated cells in buffer. During subsequent detector training, images that produced misclassifications were resurfaced in successive epochs by hard-sample mining (further details in Methods), ensuring repeated exposure to the most confounding features.
An asymmetric cross-entropy loss embodied the biological class imbalance: weights were skewed towards healthy-blood examples to penalize WBC misclassification while remaining tolerant of occasional pseudo-labeling errors in the positive stream. Further details can be found in Methods. After training, additional cancer lines—25 in total—were sequestered for validation, demonstrating that the network learns invariant features that generalize beyond the limited set of cultures available for optimization. The full list of cell lines, hyper-parameters, and augmentation protocols are provided in Methods and SI §4 Tables 3–4.
Together, the key-point formulation, HR-Net customization, heterogeneity-centered sampling, and asymmetric loss establish a conceptually distinct framework (Fig. 3, center and right panels) that narrows the gap between idealized cell-line imagery and the complex reality of patient-derived CTCs while sustaining the high frame rates required for population-scale liquid biopsy.
System Validation
To evaluate the overall detection performance of the system, we conducted validation experiments using both spiked and unspiked blood samples. Each specimen was prepared using approximately 3 mL of healthy donor whole blood. One group was spiked with LNCaP prostate cancer cells (n = 18), while the control group remained unspiked (n = 18). To enable fluorescence-based confirmation, cancer cells were stained with a cell tracker dye prior to spiking, simulating an ideal condition with uniform marker expression and minimal nonspecific labeling. The unspiked control cohort was used to determine the system’s false positive rate (FPR). In contrast, the spiked samples were used to calculate the recovery rate, defined here as the proportion of introduced cancer cells that were successfully identified by the system. This recovery rate reflects the true positive rate (TPR), also referred to as sensitivity. Full experimental protocols are described in SI §5.1.
To contextualize these results, we also generated a theoretical recovery curve based on a large validation dataset containing 25 cancer cell lines not used during training. As shown in Fig. 4a, the recovery rate increases approximately linearly as the model confidence threshold decreases—from about 30% at a threshold of 0.9 to nearly 80% at 0.2. This trend reflects the expected trade-off between sensitivity and specificity. At higher thresholds, the system is more conservative and yields fewer detections, while lower thresholds allow more potential CTCs to be captured at the cost of admitting more false positives. Across the range from approximately 0.2 to 0.8, the experimental recovery rate was consistently lower than the theoretical prediction, with a maximum deviation of approximately 10%. This discrepancy may be attributed to biological variability between the controlled cell line conditions (i.e., cell lines in pure buffer) versus the more complex background of spiked blood samples (i.e., cell lines in blood). It also reflects necessary differences in how recovery is defined for these two different conditions (further details are available in SI §5). Nonetheless, both curves showed qualitatively similar trends, indicating that the model generalizes well across different sample contexts.
Fig. 4: Model validation using spiked experiments.
a Measured recovery at different operating thresholds. b False positive rate (FPR), shown as discrete points due to the low number of false positives detected, with only 5 false positive cells found across all unspiked samples (totaling 55 mL of whole blood) at the operating threshold of 0.5.
Given the expected abundance of CTCs in late-stage cancer patients, estimated at around 10 cells/mL, an operating threshold of 0.5 was chosen. At this threshold, the system achieved a TPR of around 60% while maintaining a low FPR. Lowering the threshold would improve sensitivity but introduce more false positives, which can undermine specificity in clinical use. This balance between FPR and TPR is critical for optimizing diagnostic accuracy, especially in the context of rare cell detection. Fig. 4b shows the measured FPR across a range of thresholds, expressed in terms of false positive detections per mL in the control samples. Because false positive events were rare, the plot is composed of discrete points. At the chosen threshold of 0.5, only 5 false positives were detected in total across all healthy blood samples, corresponding to fewer than 0.1 false positives/mL (total unspiked sample volume was approximately 55 mL, i.e., slightly more than 3 mL per sample from the 18 samples). Even at the most permissive threshold of 0.1, the FPR remained below 1 cell/mL. Data concerning the model’s FPR based on nucleated cells in healthy blood without IF data can be found in SI §3.2. This analysis helps clarify how FPR behaves in the absence of orthogonal molecular labeling and supports evaluation of system performance in strictly label-free settings.
To complete the assessment of diagnostic performance, we estimated the positive predictive value (PPV, or precision), which defines the probability that a detected cell is a true positive. While our validation experiments do not permit a direct measurement of PPV, it can be calculated from the experimentally determined TPR and FPR. For this estimation, we assumed a clinically representative CTC abundance of 10 cells/mL. At the chosen operating threshold of 0.5, this model yields a PPV of approximately 0.98. Such a high PPV underscores the system’s strong specificity and indicates that detections are highly likely to be true positives, a critical feature for reliable rare cell analysis.
Patient Sample Performance
To evaluate the efficacy of our platform under clinically relevant conditions, we assessed its ability to detect CTCs in blood samples from prostate cancer patients in comparison to healthy donors. The patient cohort comprised 13 male subjects diagnosed with metastatic castration-resistant prostate cancer (mCRPC), while the healthy control cohort consisted of 8 male donors without any known cancer diagnosis. Blood samples from both cohorts were each collected as part of separate studies. As such, subsequent analysis was not strictly blinded, though significant efforts were made to analyze all samples equally. Samples were collected within the same time window for both cohorts, between November 1, 2023, and February 1, 2024. There were no differences in protocol followed by the laboratory or in subsequent data processing between cohorts, as the same microfluidic enrichment, immunofluorescent staining, data capture and CTC enumeration steps were followed.
Patient samples were subjected to staining for prostate-specific membrane antigen (PSMA) and EpCAM proteins, commonly expressed on the surface of prostate cancer cells and epithelial cells, respectively46. A cell was classified as a CTC only if it exceeded the holography model threshold of 0.5 and exhibited a positive PSMA signal, as determined by PMT scoring (see SI §2.1 for details). This dual requirement was designed to ensure high tumor specificity and reduce the likelihood of misclassifying non-malignant nucleated cells. Although EpCAM expression was also measured, it was intentionally excluded from the criteria for CTC identification. This decision reflects the well-established observation that EpCAM expression can be downregulated in CTCs undergoing epithelial-to-mesenchymal transition (EMT), a process associated with increased metastatic potential and poorer prognosis9,11. Relying on EpCAM alone would risk missing a clinically significant subset of tumor cells. Instead, EpCAM was used as a phenotypic marker to help characterize the epithelial profile of the detected PSMA-positive cells.
This molecular gating strategy revealed a substantial proportion of EpCAM-negative CTCs: across the patient cohort, only 37% of PSMA-positive cells were also EpCAM-positive. This finding has important implications. It suggests that a large fraction of CTCs would likely evade detection in conventional EpCAM-based platforms, particularly in patients with epithelial-to-mesenchymal phenotypic shifts. By leveraging label-free holographic morphology in conjunction with PSMA, the system is capable of identifying a broader spectrum of tumor cells, including EpCAM-low or EpCAM-negative populations.
Figure 5a shows the distribution of detected CTC counts across both cohorts, with values normalized to the initial whole blood volume to account for variability in sample collection. Cancer patients exhibited a markedly higher CTC burden than healthy donors, with median counts of 12.5 cells/ml and 1.5 cells/ml, respectively (counts per patient are provided in SI §6). These findings are consistent with previously reported ranges for late-stage prostate cancer (e.g.47,) and support the system’s ability to reliably distinguish clinical from non-clinical samples. The spread within the patient group reflects the expected biological heterogeneity in disease progression and tumor shedding rates. Additionally, several patients exhibited CTC counts exceeding 20 cells/ml, while healthy individuals exhibited consistently low counts with few outliers, further reinforcing diagnostic discriminability. Representative holograms of CTCs detected in patient samples are shown in Fig. 5b–e, along with corresponding IF signals in Fig. 5f–i. The detected cells display a wide range of morphological features and antigen expression profiles, including EpCAM-negative phenotypes that are often underrepresented in traditional antigen-based CTC detection assays such as CellSearch®.
Fig. 5: Detection rate of the model evaluated on a cohort of healthy donors and donors with late-stage prostate cancer (mCRPC).
a Distribution of detected CTCs in patient samples, normalized to the volume of whole blood in the initial tube to account for sample variability. b-e CTCs identified in prostate cancer patient samples, with corresponding immunofluorescent (IF) signal f–i. For the two IF channels captured, PSMA served as a prostate-specific marker and EpCAM served as a pan-cancer marker (for epithelial cells).
These findings suggest that the system’s applicability extends to the detection of patient-derived CTCs, notwithstanding its initial training using cultured cell lines. Similar to the spiked experiments, the utilization of PSMA staining effectively decreased false positive signals in healthy samples. The results from these patient samples provide critical evidence of the system’s efficacy beyond controlled lab settings and emphasizes the potential of our platform as a powerful tool for cancer diagnostics and screening.
