rplatypus Brings PyTorch Medical Imaging Workflows to R
A new architecture for medical imaging in R
rplatypus has been rebuilt around a practical gap in R: researchers can read medical images with established packages, but have had few ways to connect that work to modern deep learning without shifting their workflow to Python. The new package keeps experiment setup and image analysis in R while delegating model execution to a Python engine built on PyTorch.
The project began as platypus, an R package for object detection and segmentation using YOLOv3 and U-Net through keras and TensorFlow. After its initial 2020 release, it drew 134 GitHub stars and a growing issue queue, but eventually stopped working reliably. The maintainer has now replaced it with a narrower package focused on medical imaging. At the time of publication, rplatypus had been submitted to CRAN and was awaiting review; the GitHub version could be installed with:
remotes::install_github("maju116/rplatypus")My read is that the rewrite matters more than the new package name. It changes how the R layer reaches its models, how Python dependencies are managed, and how the project can keep pace with medical imaging research.
Why the TensorFlow version became difficult to maintain
The original package depended on R’s keras and tensorflow packages, which connect to a Python installation through reticulate. That setup could make an unrelated Python change affect an R workflow: for example, upgrading numpy for another project could break a platypus session. Since the resulting error might come from someone else’s Python environment, it was hard for the package maintainer to diagnose or fix.
There was also a dependency chain between platypus, the R interface packages and TensorFlow itself. When TensorFlow changed, the R packages needed to catch up before platypus could follow. The project’s older engine was pinned to TensorFlow 2.9, which does not install on Python 3.12. Staying with TensorFlow would therefore have meant another substantial rewrite, with little prospect of aligning the package with the tools now common in medical imaging.
Two developments made a different design possible. First, reticulate added py_require() in version 1.41, allowing packages to declare Python dependencies and use isolated environments built with uv. rplatypus can set up a private environment when its engine is first needed, then reuse it. The user doesn’t need to select an interpreter or manually activate an environment, and the package no longer depends on a shared Python installation.
Second, medical imaging’s deep learning ecosystem has converged around PyTorch. Projects and resources such as MONAI, nnU-Net, TorchIO, timm, pretrained encoders and foundation models are part of that ecosystem. The rebuilt package can draw on those Python tools through its own engine, without waiting for R wrappers around TensorFlow to catch up.
R interface, Python engine
The execution layer is pyplatypus, a Python package available from PyPI. It started in 2022 on TensorFlow and was rewritten alongside rplatypus. Python users can install it directly with:
pip install "pyplatypus[hub]"The division of work is deliberate. PyTorch models, pretrained weights and medical imaging libraries live in Python, where the engine can use them directly. R handles the tasks around model execution: reading and combining masks, plotting, comparing results and analyzing experiments. The maintainer considered native R torch, which binds to libtorch without Python, but maintaining architectures, losses and data pipelines in both languages would be a large burden for a one-person project. It would also cut the package off from pretrained weights in Python’s ecosystem.
The two packages share a release sequence. Features are implemented in pyplatypus, published to PyPI, then adopted by rplatypus. Each R release pins an exact engine version, reducing the chance that an engine change will break an existing R package release. Both sides also read the same YAML configuration format, so an experiment can move between R and Python without being rewritten.
Packaging work made the rewrite feasible
The maintainer credits an AI coding agent, Claude Code, with helping complete the rewrite in about a month. The project’s purpose, datasets, model requirements and acceptance criteria remained human decisions; reported results were measured from actual runs. The agent helped with supporting work such as tests, documentation, coordination between the R and Python packages, and debugging across reticulate, uv and CUDA.
That distinction is relevant to the package’s technical claims. In machine learning, a result is only useful when the experiment and its evaluation can be inspected. The maintainer says the measurements in the planned series were run rather than generated, and the package reports limitations alongside available pretrained weights.
A pretrained segmentation example
The simplest demonstration uses a published U-Net model trained on the 2018 Data Science Bowl nuclei dataset. The example uses four microscopy images stored in the dataset’s directory structure and loads the pretrained weights without fitting a new model:
library(rplatypus)
spec <- platypus_spec(
data = segmentation_data("nuclei", "nuclei", colormap = binary_colormap),
models = list(u_net(
"nuclei",
input_shape = c(256, 256),
blocks = 4,
filters = 16,
weights = "dsbowl-unet",
fit = FALSE
))
)
fit <- platypus_fit(spec, verbose = FALSE)
masks <- predict(fit, split = "validation")
dim(masks)
#> [1] 4 256 256The result contains one 256-by-256 mask per image, with a class label for each pixel. The example ran on a CPU; platypus_device() reports whether a GPU is available and explains when it can’t be used.
dsbowl-unet weights on four microscopy images.The first three images are fluorescence microscopy, where the predicted nuclei appear as distinct regions. The fourth is a brightfield image of stained cells; there, the model detects small dark nuclei. The example is useful as a visual check, but it isn’t a fair evaluation: these images come from the same collection used to train the weights. The maintainer deliberately provides no score for them.
For evaluation, available_weights() describes the reported held-out results and the limits of each model. One example is the semantic nature of this U-Net: adjacent nuclei may merge into one predicted region. A count derived from those masks could therefore be wrong even when the segmented area looks plausible. That’s the sort of distinction that a single aggregate score can conceal.
One experiment specification across languages
An experiment can be defined with R arguments or stored in YAML. The example configuration records the segmentation task, paths, colormap, architecture, input dimensions and pretrained weights:
task: semantic_segmentation
data:
train_path: nuclei
validation_path: nuclei
colormap: [[0, 0, 0], [255, 255, 255]]
models:
- name: nuclei
architecture: u_net
input_shape: [256, 256]
blocks: 4
filters: 16
weights: dsbowl-unet
fit: falseLoading this file into R produces the same specification as the code-built object. The Python engine can read the same file and load the same weights:
from pyplatypus import build_engine, from_yaml
engine = build_engine(from_yaml("nuclei.yaml"))
engine.fit()
masks = engine.predict("nuclei", split="validation")With fit: false, the call loads the published weights rather than training a model. This shared format gives R and Python users a concrete handoff: they can work in their preferred language while describing the same experiment.
What the package supports today
rplatypus covers several common medical imaging tasks. Segmentation includes 2D and 3D U-Net, U-Net++, Res-U-Net and LinkNet architectures, eleven loss functions and five metrics that can exclude the background. Excluding background can matter when it occupies most of an image and would otherwise dominate a score.
Classification reports AUC rather than relying on accuracy alone. In an imbalanced dataset where only 3% of cases have a finding, a model that always predicts “no finding” can reach 0.97 accuracy while detecting nothing. Object detection uses YOLOv3 with anchors fitted to the dataset’s boxes rather than borrowed from COCO, and instance segmentation supports predictions that represent individual objects rather than only regions.
For medical volumes, the package accepts NIfTI and DICOM data, accounts for voxel spacing, supports splits by patient rather than by slice, and reports volumes in millilitres. It can also report scores per case so that poor results can be traced to specific images. Before training, it checks for issues such as masks that don’t match the declared colormap, channel ordering errors and missing slices in a DICOM series.
The reference documentation and seven vignettes, six based on precomputed runs, are available at maju116.github.io/rplatypus. The current model set is a starting point. The maintainer lists multi-label classification, ensembling and 3D boxes among areas still missing, with future priorities partly dependent on user needs.
Measurements ahead
The planned series uses specific datasets to examine where model scores help and where they mislead. Topics include a nuclei model with a mean Dice score of 0.855 that scored 0.006 on three images; a classifier with 0.97 accuracy that learned nothing useful; YOLOv3 results where a one-point mAP difference across five seeds may be noise; and CT segmentations with a Dice score of 0.88 despite volume estimates 12–23% too large.
Other planned analyses cover retinal vessels that lose detail when resized, counts of touching nuclei that vary between 245 and 180 from the same forward pass, and pretrained encoders measured across three imaging modalities. In the reported comparison, an encoder with 1.9 million parameters beats ResNet-34 on two of the three modalities.
These examples point to the package’s central value: connecting R’s medical image handling and analysis tools to PyTorch models, while keeping configuration portable and evaluation tied to individual cases. The GitHub release is the current source for the latest version; CRAN submissions are expected less frequently, typically at intervals of a month or two.