Search ErnesTech - ErnesTech

Computer Vision in PyTorch (Part 2) Preparing Dat ...

In Part 1 of this tutorial series, we explored the fundamentals of Convolutional Neural Networks (CNNs) and built a complete CNN architecture using PyTorch for pneumonia detection in chest X-rays. We learned why CNNs excel at image tasks, examined each component in detail, and implemented a custom PneumoniaCNN class by taking an OOP approach and subclassing PyTorch's nn.Module class. Now it's time to bring our model to life! In this tutorial, we'll complete our pneumonia detection system by Preparing and preprocessing the chest X-ray dataset Training our CNN model with a complete training loop Evaluating model performance using metrics like precision, recall, and F1 Interpreting evaluation results with a focus on visualizing predictions Addressing common CNN training issues like overfitting, underfitting, and class imbalance By the end of this tutorial, you'll have transformed your CNN architecture into a working medical diagnostic tool and gained practical skills for implementing and evaluating deep learning models. Let's get started! Prerequisites Before proceeding, make sure you've Read Part 1 of this tutorial series Installed PyTorch (follow PyTorch's official installation instructions) Reviewed fundamental deep learning concepts such as Layers Activation functions Loss Optimization Below are the modules, functions, and classes we’ll need for this tutorial. Be sure to install any libraries you’re missing after running this code import torch import torch.nn as nn import torch.nn.functional as F import torch.optim as optim from torch.utils.data import DataLoader, Dataset import torchvision import torchvision.transforms as transforms from torchvision.datasets import ImageFolder import matplotlib.pyplot as plt import numpy as np import tarfile import os import collections import random from sklearn.metrics import confusion_matrix, classification_report from sklearn.model_selection import train_test_split from PIL import Image import seaborn as sns 1. Preparing and Preprocessing the X-ray Image Dataset We’ll start by preparing our chest X-ray dataset. The dataset contains X-ray images of lungs classified into two categories NORMAL and PNEUMONIA. Downloading and Extracting the Dataset The data for this tutorial is available for download here as a compressed tar.gz file. After downloading, you'll need to extract it to access the images # Path to the downloaded tar.gz file dataset_path = "xray_dataset.tar.gz" # If saved to your current directory # Extract the dataset with tarfile.open(dataset_path, "rgz") as tar tar.extractall() print("Dataset extracted successfully") After extraction, you should have this directory structure chest_xray/ +-- test/ ¦ +-- NORMAL/ ¦ +-- PNEUMONIA/ +-- train/ +-- NORMAL/ +-- PNEUMONIA/ Verifying Dataset Structure and File Counts After extracting the dataset, it's good practice to verify the contents and get a count of the image files. This ensures we're working with the correct data and helps identify potential issues early on. We'll create a small helper function to scan the train and test directories. This function will gather the file paths for all valid JPEG images and their corresponding class labels (0 for NORMAL, 1 for PNEUMONIA). Collecting these paths and labels now will also prepare us for the next step in preparing our data for training. # Define base directories relative to your notebook/script location data_dir = "chest_xray" train_dir = os.path.join(data_dir, "train") test_dir = os.path.join(data_dir, "test") # Define the classes based on the subfolder names class_names = ['NORMAL', 'PNEUMONIA'] class_to_idx = {cls_name i for i, cls_name in enumerate(class_names)} # Helper function to scan directories, filter JPEG images, and collect paths/labels def get_image_paths_and_labels(data_dir) image_paths = [] labels = [] print(f"Scanning directory {data_dir}") for label_name in class_names class_dir = os.path.join(data_dir, label_name) count = 0 # List files in the class directory for filename in os.listdir(class_dir) # Keep only files ending with .jpeg (case-insensitive) if filename.lower().endswith('.jpeg') image_paths.append(os.path.join(class_dir, filename)) labels.append(class_to_idx[label_name]) count += 1 print(f" Found {count} '.jpeg' images for class '{label_name}'") return image_paths, labels # Get paths and labels for the training set all_train_paths, all_train_labels = get_image_paths_and_labels(train_dir) train_counts = collections.Counter(all_train_labels) total_train_images = len(all_train_paths) print(f"Training Set Counts") print(f" NORMAL (Class 0) {train_counts[class_to_idx['NORMAL']]}") print(f" PNEUMONIA (Class 1) {train_counts[class_to_idx['PNEUMONIA']]}") print(f" Total Training Samples {total_train_images}") # Get paths and labels for the test set all_test_paths, all_test_labels = get_image_paths_and_labels(test_dir) test_counts = collections.Counter(all_test_labels) total_test_images = len(all_test_paths) print(f"Test Set Counts") print(f" NORMAL (Class 0) {test_counts[class_to_idx['NORMAL']]}") print(f" PNEUMONIA (Class 1) {test_counts[class_to_idx['PNEUMONIA']]}") print(f" Total Test Samples {total_test_images}") Running this code will scan the directories and produce the following counts Scanning directory chest_xray/train Found 1349 '.jpeg' images for class 'NORMAL' Found 3883 '.jpeg' images for class 'PNEUMONIA' Training Set Counts NORMAL (Class 0) 1349 PNEUMONIA (Class 1) 3883 Total Training Samples 5232 Scanning directory chest_xray/test Found 234 '.jpeg' images for class 'NORMAL' Found 390 '.jpeg' images for class 'PNEUMONIA' Test Set Counts NORMAL (Class 0) 234 PNEUMONIA (Class 1) 390 Total Test Samples 624 Excellent! Our helper function scanned the directories and gave us clean lists of all the usable JPEG images and their corresponding labels for both training and testing (stored in variables like all_train_paths, all_train_labels, etc.). Now, looking at the training counts (1349 NORMAL to 3883 PNEUMONIA), something immediately stands out there are almost three times as many pneumonia examples in our training data! This situation, where one class significantly outnumbers another, is called class imbalance. While techniques exist to directly address class imbalance during training (we’ll talk about those later), our plan for now is to first train the model using the data as-is. That said, this imbalance means we'll need to be especially careful when we get to evaluating the model's performance. We can't just rely on overall accuracy; we'll need to use specific metrics that tell us how well the model identifies both classes fairly. But we’ll get to that too. Having prepared the lists of training image paths and labels, we're now ready for the next important step in preparing our data splitting off a portion of the training images to create a validation set. Why You Need a Validation Set Now that we have lists of our training images and associated labels, you might be thinking, “Why did we need those specific lists?” Well, before we train our model, we need to set aside a portion of that training data to create a validation set. It might seem strange to not use all available data for training, but this validation split is vital for trustworthy model development. Here's why Tuning & Monitoring While training, we need to monitor how well the model is learning and potentially tune things like the learning rate or decide when to stop training. We need a dataset for this that the model isn't directly training on, but which isn't our final, untouched test set. That's the validation set's job. Avoiding Data Leakage If we used the test set to make these tuning decisions, we'd essentially be "leaking" information about the test set into our model development process. The model might end up looking good on that specific test set simply because we optimized for it, but fail to generalize to new, truly unseen data. Unbiased Final Test The test set should only be used once, at the very end, after all training and tuning are complete, to get an unbiased estimate of the final model's performance. So, we reserve the test_paths/test_labels for the final evaluation and split our all_train_paths/all_train_labels into two new subsets one for actual training, and one for validation during development. We'll use the train_test_split function from scikit-learn for this. Because we identified a class imbalance earlier, we'll use the stratify option to ensure both the new training subset and the validation set maintain the original proportion of NORMAL and PNEUMONIA images. Using random_state ensures the split is the same every time the code runs. # Define proportion for validation set val_split_ratio = 0.2 SEED = 42 # Perform stratified split train_paths, val_paths, train_labels, val_labels = train_test_split( all_train_paths, all_train_labels, test_size=val_split_ratio, stratify=all_train_labels, random_state=SEED ) # Print the number of samples in each resulting set print(f"Original training image count {len(all_train_paths)}") print(f"--> Split into {len(train_paths)} training samples") print(f"--> Split into {len(val_paths)} validation samples") Running this will perform the split and show the resulting counts Original training image count 5232 --> Split into 4185 training samples --> Split into 1047 validation samples We now have distinct lists of file paths and corresponding labels for our training data (train_paths, train_labels) and our validation data (val_paths, val_labels). These lists tell us which images belong in each set. But simply having file paths isn't enough to feed data into a PyTorch model. Each image needs to be loaded and undergo several processing steps first. These include standard operations like resizing all images to a consistent 256×256 dimension and converting them into the correct format (single-channel grayscale tensors). Additionally, to help our model learn more robust features from a smaller dataset and generalize better from our specific training images, we'll apply a technique called data augmentation, but only on the training set. Understanding Data Augmentation We have our training images identified, but deep learning models often benefit from seeing a large variety and quantity of data. What if our training set, particularly after splitting, isn't large or diverse enough to teach the model to generalize well to all possible variations it might encounter in new X-rays? This is where data augmentation comes in. What is Data Augmentation? Data augmentation is a technique used to artificially increase the diversity of your training dataset without actually collecting new images. It involves applying random, yet realistic, transformations to the images during the training process. Each time the model sees an image from the training set, it might see a slightly altered version (e.g., flipped horizontally or slightly rotated). Why Use Data Augmentation? Improved Generalization & Robustness By exposing the model to these variations (like different orientations or flips), it learns to focus on the underlying patterns relevant to the task (e.g., signs of pneumonia) rather than potentially irrelevant characteristics like the exact positioning of the patient. This helps the model generalize better to new, unseen images that might have similar slight variations. Reduced Overfitting It effectively increases the perceived size of the training set, making it harder for the model to simply memorize the training examples. This is particularly valuable when working with specialized datasets (like medical images) that might be smaller than general-purpose image datasets. Our Chosen Augmentations For this tutorial, we'll apply two simple and common augmentation techniques using torchvision.transforms transforms.RandomHorizontalFlip(0.5) This randomly flips the image horizontally (left-to-right) with a default probability of 50%. transforms.RandomRotation(10) This randomly rotates the image by a small angle, in this case, up to 10 degrees in either direction. These simple variations help the model learn features that aren't dependent on perfect orientation or specific left-right positioning. Many other augmentation techniques exist, like adjusting brightness/contrast, zooming, or shearing, but we'll stick to these two for now. Important Training Only! Crucially, data augmentation is applied only to the training set. We do not apply random augmentations to the validation or test sets. Why? Because we need a consistent and unbiased measure of the model's performance on unmodified data during validation (for tuning) and testing (for final evaluation). Augmenting validation/test data would introduce randomness that makes performance measurement unreliable. Now that we understand the concept and benefits of data augmentation, let's define the complete image transformation pipelines for our training, validation, and test sets. Defining Image Transforms Now that we have lists specifying which images belong to our training and validation sets, we need to define how to process each image file into a standardized tensor format suitable for our PyTorch model. This involves creating processing pipelines using torchvision.transforms. We'll need slightly different pipelines for training data (which includes random augmentation) and for validation/test data (which does not). These pipelines need to perform several key operations consistently for every image Resize to Fixed Dimensions To meet our model's required 256×256 input size, the first step is transforms.Resize((256, 256)). Be aware that because the original X-rays vary in size, this forces a square aspect ratio and will distort non-square images by stretching or squashing them. While this could potentially obscure subtle diagnostic cues related to shape or proportion, CNNs can often adapt and learn effectively from such consistently distorted data. We'll use this standard resizing approach for our fixed-input model, but keep in mind that if evaluation reveals performance issues potentially linked to shape distortion, exploring aspect-preserving alternatives (like padding before resizing) would be a logical next step to investigate. Ensure Grayscale The model architecture also expects single-channel grayscale images (in_channels=1). To guarantee this format for all images processed, we include transforms.Grayscale(num_output_channels=1). Data Augmentation (Training Only) For the training pipeline (train_transforms), we'll insert the transforms.RandomHorizontalFlip() and transforms.RandomRotation(10) steps discussed in the previous section to help the model generalize better. These are not included in the validation/test pipeline. Convert to Tensor & Scale The final step is transforms.ToTensor(). This performs two critical functions it converts the processed PIL Image object into a PyTorch tensor, and it scales the pixel values from the original integer range [0, 255] down to a floating-point range of [0.0, 1.0]. This [0, 1] scaling acts as our input normalization for this tutorial. We are opting for this simpler approach instead of standardizing with a separate transforms.Normalize(mean, std) step, relying partly on the BatchNorm2d layers within our model to help adapt to the input distribution during training. Creating the Pipelines With these steps decided, we define two distinct data process pipelines using transforms.Compose # Transformations for the training set (including augmentation) train_transforms = transforms.Compose([ transforms.Resize((256, 256)), transforms.Grayscale(num_output_channels=1), transforms.RandomHorizontalFlip(), transforms.RandomRotation(10), transforms.ToTensor() # Converts to tensor AND scales to [0, 1] ]) # Transformations for the validation and test sets (NO augmentation) val_test_transforms = transforms.Compose([ transforms.Resize((256, 256)), transforms.Grayscale(num_output_channels=1), transforms.ToTensor() # Converts to tensor AND scales to [0, 1] ]) print("Transformation pipelines defined.") Now that we've defined how to process the images with train_transforms and val_test_transforms, we need an efficient way to connect these pipelines to our lists of image paths (train_paths, val_paths, etc.). Specifically, we need a structure that can take an index, find the corresponding image path and label, load the image file, apply the correct transformations, and provide the resulting tensor and label to PyTorch for training or evaluation. This requires creating a custom PyTorch Dataset. Let's build that next. Creating a Custom PyTorch Dataset We have our lists of image paths, and we have our processing pipelines, so now let’s bring them together so PyTorch can load and transform images during training and evaluation. While PyTorch offers built-in datasets like ImageFolder, they assume a specific directory structure and aren't ideal for using pre-split lists of file paths with different transforms assigned to each split. Thankfully, PyTorch makes it straightforward to create our own custom dataset handling logic by inheriting from the base torch.utils.data.Dataset class. A custom Dataset needs to implement three essential methods __init__(self, ...) Initializes the dataset, typically by storing file paths, labels, and any necessary transformations. __len__(self) Returns the total number of samples in the dataset. __getitem__(self, idx) Loads and returns a single sample (usually an image tensor and its label) from the dataset, given an index idx. This method is where the image loading and transformations are actually applied, often "just-in-time" when the sample is requested. Let's define our XRayDataset class class XRayDataset(Dataset) """Custom Dataset for loading X-ray images from file paths.""" def __init__(self, image_paths, labels, transform=None) """ Args image_paths (list) List of paths to images. labels (list) List of corresponding labels (0 or 1). transform (callable, optional) Optional transform to be applied on a sample. """ self.image_paths = image_paths self.labels = labels self.transform = transform def __len__(self) """Returns the total number of samples in the dataset.""" return len(self.image_paths) def __getitem__(self, idx) """ Fetches the sample at the given index, loads the image, applies transformations, and handles potential errors. Args idx (int) The index of the sample to fetch. Returns tuple (image_tensor, label) if successful. None If an error occurs (e.g., file not found, processing error), signalling to skip this sample. """ # Get the path and label for the requested index img_path = self.image_paths[idx] label = self.labels[idx] try # Load the image using PIL within a context manager with Image.open(img_path) as img # Apply transforms ONLY if they exist if self.transform # Apply the entire transform pipeline image_tensor = self.transform(img) # Return the processed tensor and label return image_tensor, label else # This branch indicates a setup error, as the transform # pipeline should at least contain ToTensor(). raise ValueError(f"Dataset initialized without transforms for {img_path}. " "Transforms (including ToTensor) are required.") except FileNotFoundError # Handle cases where the image file doesn't exist print(f"Warning Image file not found at {img_path}. Skipping sample {idx}.") return None # Returning None signals to skip except ValueError as e # Catch the specific error we raised for missing transforms print(f"Error for sample {idx} at {img_path} {e}") raise e # Re-raise critical setup errors except Exception as e # Catch any other PIL loading or transform errors print(f"Warning Error processing image {img_path} (sample {idx}) {e}. Skipping sample.") return None # Returning None signals to skip Explanation __init__ The constructor (__init__) is straightforward. It simply stores the essential information passed when we create an instance of XRayDataset the list of image paths, the corresponding list of labels, and the specific torchvision.transforms pipeline that should be applied to images from this dataset. __len__ This method allows PyTorch code to easily get the total size of the dataset by simply returning the number of image paths provided during initialization. __getitem__ This is the core method where the actual data loading and processing happens for a single sample. When requested by its index (idx), it performs the following steps Retrieves the image file path and label using the index. Opens the image file using the PIL library. Applies the entire transformation pipeline (like train_transforms or val_test_transforms) stored in self.transform. Returns the processed image tensor and its integer label if successful. Crucially, this loading and transforming happens "on demand" or "lazily." The implementation also includes basic error handling if an image file is missing or fails during processing, it prints a warning and returns None, signaling that this sample should be skipped. This XRayDataset class gives us a blueprint for handling our image data. With this class defined, we can now create the specific Dataset instances we need one for our training data using train_paths and train_transforms, one for validation using val_paths and val_test_transforms, and one for our test set. Let's instantiate these datasets next. Creating Final Datasets and DataLoader Objects With our XRayDataset class ready, we can now instantiate it for each of our data splits. We'll pair the appropriate lists of image paths and labels with the corresponding transformation pipelines we defined earlier. # Instantiate the custom Dataset for each split train_dataset = XRayDataset( image_paths=train_paths, labels=train_labels, transform=train_transforms # Apply training transforms (incl. augmentation) ) val_dataset = XRayDataset( image_paths=val_paths, labels=val_labels, transform=val_test_transforms # Apply validation transforms (no augmentation) ) test_dataset = XRayDataset( image_paths=all_test_paths, # Using all_test_paths from verification step labels=all_test_labels, # Using all_test_labels from verification step transform=val_test_transforms # Apply validation/test transforms ) # Print dataset sizes to confirm print("Final Dataset objects created") print(f" Training dataset size {len(train_dataset)}") print(f" Validation dataset size {len(val_dataset)}") print(f" Test dataset size {len(test_dataset)}") This gives us three Dataset objects, each knowing how to access and transform its specific set of images. Final Dataset objects created Training dataset size 4185 Validation dataset size 1047 Test dataset size 624 Introducing DataLoader While Dataset objects allow us to access individual processed samples via dataset[index], we typically train neural networks on mini-batches of data, not one sample at a time. Processing batches is more computationally efficient and helps stabilize the learning process. PyTorch's torch.utils.data.DataLoader class is designed precisely for this. It takes a Dataset object and provides an iterable that yields batches of data. Key features include Batching Automatically groups individual samples from the Dataset into batches of a specified size (batch_size). Shuffling Can automatically shuffle the training data at the beginning of each epoch (shuffle=True) to ensure the model doesn't learn based on the order of examples. Shuffling is typically disabled for validation and testing for consistent evaluation. Parallel Loading Can use multiple background worker processes (num_workers) to load data concurrently, preventing data loading from becoming a bottleneck during training, especially when using a GPU. The num_workers argument specifies how many subprocesses to use for data loading. While values > 0 can speed things up by loading data in parallel, they can sometimes cause issues in certain environments (like Colab notebooks). If you encounter errors during training related to workers, try setting num_workers=0, which loads data in the main process. Memory Pinning Can use pin_memory=True to speed up data transfer from CPU to GPU memory when training on CUDA-enabled devices. Creating the DataLoader Instances Let's create DataLoader instances for each of our datasets # Define batch size (can be tuned depending on GPU memory) batch_size = 32 # Create DataLoader for the training set train_loader = DataLoader( dataset=train_dataset, batch_size=batch_size, shuffle=True, # Shuffle data each epoch for training num_workers=2, # Number of subprocesses to use for data loading (adjust based on system) pin_memory=True # Speeds up CPU-GPU transfer if using CUDA ) # Create DataLoader for the validation set val_loader = DataLoader( dataset=val_dataset, batch_size=batch_size, shuffle=False, # No need to shuffle validation data num_workers=2, pin_memory=True ) # Create DataLoader for the test set test_loader = DataLoader( dataset=test_dataset, batch_size=batch_size, shuffle=False, # No need to shuffle test data num_workers=2, pin_memory=True ) print(f"DataLoaders created with batch size {batch_size}.") With train_loader, val_loader, and test_loader created, our data preparation pipeline is complete! These loaders are now ready to efficiently supply batches of preprocessed image tensors and labels to our model during the training, validation, and testing phases. A good next step is often to visualize a few images from the train_loader to visually inspect the results of the transformations and augmentations before proceeding to model training. Visualizing Sample Images Before we start training, it's crucial to visually inspect the output of our DataLoader objects. This acts as a sanity check to ensure our data loading, preprocessing, and augmentation steps are working correctly – essentially, we get to "see what the model will see." Let's create a helper function to display a batch of images def show_batch(dataloader, class_names, title="Sample Batch", n_samples=8) """Displays a batch of transformed images from a DataLoader.""" try images, labels = next(iter(dataloader)) # Get one batch except StopIteration print("DataLoader is empty or exhausted.") return # Limit number of samples to display if batch is smaller than n_samples actual_samples = min(n_samples, images.size(0)) if actual_samples <= 0 print("No samples found in the batch to display.") return images = images[actual_samples] labels = labels[actual_samples] # Tensors are likely on GPU if device='cuda', move to CPU for numpy/plotting images = images.cpu() labels = labels.cpu() # Determine subplot layout if actual_samples <= 4 ncols = actual_samples; nrows = 1; figsize = (3 * ncols, 4) else ncols = 4; nrows = 2; figsize = (12, 6) fig, axes = plt.subplots(nrows=nrows, ncols=ncols, figsize=figsize) if nrows == 1 and ncols == 1 axes = np.array([axes]) # Handle single plot case axes = axes.flatten() # Flatten axes array for easy iteration fig.suptitle(title, fontsize=16) for i in range(actual_samples) ax = axes[i] img_tensor = images[i] # Shape is [C=1, H, W], scaled [0.0, 1.0] # Reminder ToTensor scaled pixels to [0, 1] # Matplotlib can directly display tensors in this range with cmap='gray' # Permute dimensions from [C, H, W] to [H, W, C] for matplotlib img_display = img_tensor.permute(1, 2, 0).numpy() # Display the image, removing the channel dimension using squeeze() for grayscale # Specify vmin/vmax ensures correct display range for float data ax.imshow(img_display.squeeze(), cmap='gray', vmin=0.0, vmax=1.0) ax.set_title(f"Class {class_names[labels[i]]}") # Use passed class_names ax.axis('off') # Hide any unused subplots if the grid is larger than needed for j in range(actual_samples, len(axes)) axes[j].axis('off') plt.tight_layout() plt.subplots_adjust(top=0.88 if title else 0.95, hspace=0.3) # Adjust for suptitle plt.show() # Visualize training samples (should show augmentations) print("Visualizing a batch from train_loader...") show_batch(train_loader, class_names, title="Sample Processed Training Images") # Visualize validation samples (should NOT show augmentations) print("Visualizing a batch from val_loader...") show_batch(val_loader, class_names, title="Sample Processed Validation Images") Visualizing a batch from train_loader... Visualizing a batch from val_loader... Interpreting the Visualizations The images displayed above are samples drawn directly from our train_loader and val_loader. They reflect the full preprocessing pipeline Resized to 256×256 pixels. Converted to single-channel grayscale. If from train_loader Randomly flipped horizontally and/or slightly rotated due to data augmentation. Converted to PyTorch tensors with pixel values scaled to the [0.0, 1.0] range via transforms.ToTensor. What to Look For Format You should see grayscale chest X-rays, all uniformly sized. Labels Each image should have the correct class title ('NORMAL' or 'PNEUMONIA'). Augmentation Images from train_loader might show random variations (flips, rotations) each time you run this visualization. Images from val_loader should appear consistent without these random effects. Intensity Range The images are displayed directly from the [0, 1] scaled tensors. Ensure they look reasonable (not all black or all white, details visible). Orientation Marker & Augmentation You'll likely notice a letter marker, commonly an 'R', often placed in an upper corner of the X-rays. This marker indicates the patient's right side. Since standard chest X-rays are taken with the patient facing the detector, their right side appears on the left side of the image. Now, look closely at the samples from the train_loader if RandomHorizontalFlip was applied to an image, you’ll see this 'R' marker appearing reversed and on the right side of the image! This is a perfect visual confirmation that your training data augmentation is active. Images from the val_loader should consistently show the marker in its standard position (patient's right on the image's left). This visualization step confirms that our data loaders are correctly yielding processed image tensors in the format and range our model expects, with augmentations applied appropriately. With this confirmation, our data is ready for the main event training the CNN model. 2. Training Our CNN Model With our data prepared and loaded efficiently using DataLoader objects, we're ready to move on to the main event training the model to distinguish between NORMAL and PNEUMONIA chest X-rays. For this, we'll use the PneumoniaCNN architecture we carefully designed together previously. Instantiating the Model and Setting the Device The first step is to use the PneumoniaCNN class definition we built in Part 1 of this tutorial series. You'll need to make sure that Python class definition is available in your current environment, typically by copying the class PneumoniaCNN(nn.Module) ... block into a code cell and running it here if you haven't already. Once the PneumoniaCNN class is defined, we can create an instance of it. Don’t forget that we must then immediately move this model instance to the appropriate computing device (cpu or cuda) that we set up earlier. Performing operations between the model and data requires them both to reside on the same device. # Instantiate the model model = PneumoniaCNN() # Check if CUDA (GPU support) is available, otherwise use CPU device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # Move the model to the chosen device (GPU or CPU) model.to(device) print(f"Model '{type(model).__name__}' instantiated and moved to '{device}'.") Now our model object is created and resides on the correct device. Before we can start the training loop itself, we need two more key components A Loss Function To measure how inaccurate the model's predictions are compared to the true labels. An Optimizer To define the algorithm used to update the model's weights based on the calculated loss. Let's define these next. Defining the Loss Function and Optimizer With our model instantiated and placed on the correct device, we need two final components before building the training loop Loss Function This measures how far the model's predictions (logits) are from the actual target labels. The computed loss value is what the model tries to minimize during training. Optimizer This implements an algorithm (like Stochastic Gradient Descent or variations thereof) that updates the model's weights based on the gradients computed during the backward pass, aiming to reduce the loss. Let's define these for our task # Define loss function criterion = nn.CrossEntropyLoss() # Define optimizer optimizer = optim.Adam(model.parameters(), lr=0.0001) print("Loss function and optimizer defined.") Explanation Loss Function (criterion) We instantiate nn.CrossEntropyLoss. This is the standard choice for multi-class classification problems like ours (Normal vs. Pneumonia). It's particularly convenient because it expects the raw, unnormalized scores (logits) directly from the model's final layer and internally applies the necessary calculations (like LogSoftmax and Negative Log-Likelihood loss) to determine the error. Optimizer (optimizer) We select optim.Adam, a very popular and often effective optimization algorithm. It's known for its adaptive learning rate capabilities, meaning it can adjust the learning rate for each parameter during training, which frequently leads to faster convergence compared to simpler optimizers like basic SGD. model.parameters() We pass this to the optimizer to tell it exactly which tensors within our model are the learnable weights and biases that it should be updating. lr=0.0001 This argument sets the initial learning rate. It's a crucial hyperparameter controlling how large the updates to the weights are on each step. A value between 0.001 and 0.0001 is often a good starting point for the Adam optimizer, but it might need tuning later. Alright, all the preparatory pieces are in place! We have our instantiated model ready on the correct device, our DataLoaders (train_loader, val_loader, test_loader) prepared to serve batches of processed data, our criterion defined to measure loss, and our optimizer configured to update the model's parameters. We're finally ready to orchestrate the actual learning process by implementing the training loop. Implementing the Training Loop Now let's implement a complete training loop def train_model(model, train_loader, val_loader, criterion, optimizer, device, num_epochs=20) """Trains and validates the model.""" # Initialize lists to track metrics train_losses = [] val_losses = [] train_accuracies = [] val_accuracies = [] print("Starting Training...") # Training loop for epoch in range(num_epochs) # Training Phase model.train() # Set model to training mode (enables dropout, batch norm updates) running_loss = 0.0 correct_train = 0 total_train = 0 # Iterate over training data for i, (images, labels) in enumerate(train_loader) # Move data to the specified device images, labels = images.to(device), labels.to(device) # Zero the parameter gradients optimizer.zero_grad() # Forward pass outputs = model(images) loss = criterion(outputs, labels) # Backward pass and optimize loss.backward() optimizer.step() # Track training loss and accuracy running_loss += loss.item() * images.size(0) # loss.item() is the avg loss per batch _, predicted = torch.max(outputs.data, 1) total_train += labels.size(0) correct_train += (predicted == labels).sum().item() # Calculate training statistics for the epoch epoch_train_loss = running_loss / len(train_loader.dataset) epoch_train_acc = correct_train / total_train train_losses.append(epoch_train_loss) train_accuracies.append(epoch_train_acc) # Validation Phase model.eval() # Set model to evaluation mode (disables dropout, uses running stats for batch norm) val_loss = 0.0 correct_val = 0 total_val = 0 # Disable gradient calculations for validation with torch.no_grad() for images, labels in val_loader images, labels = images.to(device), labels.to(device) outputs = model(images) loss = criterion(outputs, labels) val_loss += loss.item() * images.size(0) _, predicted = torch.max(outputs.data, 1) total_val += labels.size(0) correct_val += (predicted == labels).sum().item() # Calculate validation statistics for the epoch epoch_val_loss = val_loss / len(val_loader.dataset) epoch_val_acc = correct_val / total_val val_losses.append(epoch_val_loss) val_accuracies.append(epoch_val_acc) # Print statistics for the epoch print(f"Epoch {epoch+1}/{num_epochs}") print(f" Train Loss {epoch_train_loss.4f}, Train Acc {epoch_train_acc.4f}") print(f" Val Loss {epoch_val_loss.4f}, Val Acc {epoch_val_acc.4f}") print("-" * 30) print("Finished Training.") # Return performance history return { 'train_losses' train_losses, 'train_accuracies' train_accuracies, 'val_losses' val_losses, 'val_accuracies' val_accuracies } This training function Tracks performance metrics (training and validation) over time. Switches between training (model.train()) and evaluation (model.eval()) modes correctly. Handles device placement for tensors (.to(device)). Implements the full train-validate cycle for each epoch. Returns a dictionary of training and validation history for later analysis. Now, let's start training our model # Train the model num_epochs = 20 history = train_model( model=model, train_loader=train_loader, val_loader=val_loader, criterion=criterion, optimizer=optimizer, device=device, num_epochs=num_epochs ) During training, you'll see output showing the model's progress Starting Training... Epoch 1/20 Train Loss 0.5181, Train Acc 0.8282 Val Loss 0.1428, Val Acc 0.9484 ------------------------------ Epoch 2/20 Train Loss 0.2066, Train Acc 0.9221 Val Loss 0.0897, Val Acc 0.9685 ------------------------------ Epoch 3/20 Train Loss 0.1632, Train Acc 0.9379 Val Loss 0.0708, Val Acc 0.9780 ------------------------------ ... (output for subsequent epochs) ... ------------------------------ Epoch 20/20 Train Loss 0.0832, Train Acc 0.9699 Val Loss 0.0468, Val Acc 0.9819 Finished Training. Training is complete! The output above gives us a snapshot of the loss and accuracy progress for each epoch on both the training and validation sets. We can see the model is learning, but to get a full picture of the trends over all 20 epochs, like how quickly the model converged, whether overfitting occurred, and how the validation performance truly compared to training, we should visualize these metrics instead. So let's plot the history next. Visualizing the Training Process Visualizing the training and validation metrics is the best way to understand how our model is learning. Plotting their loss/accuracy curves over epochs provides valuable insights into the learning dynamics. def plot_training_history(history) """Plots the training and validation loss and accuracy.""" fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(15, 5)) # Plot losses ax1.plot(history['train_losses'], label='Train Loss') ax1.plot(history['val_losses'], label='Validation Loss') ax1.set_xlabel('Epoch') ax1.set_ylabel('Loss') ax1.set_title('Training and Validation Loss') ax1.legend() ax1.grid(True) # Plot accuracies ax2.plot(history['train_accuracies'], label='Train Accuracy') ax2.plot(history['val_accuracies'], label='Validation Accuracy') ax2.set_xlabel('Epoch') ax2.set_ylabel('Accuracy') ax2.set_title('Training and Validation Accuracy') ax2.legend() ax2.grid(True) plt.tight_layout() plt.show() # Plot the training and validation history plot_training_history(history) These plots provide a clear visual summary of the entire training process over the 20 epochs. Overall Learning We can clearly see the learning trend both the blue (training) and orange (validation) loss curves decrease significantly from the start and then begin to level off, particularly towards the end. Correspondingly, both accuracy curves rise quickly and plateau at a high level. This confirms the model successfully learned from the data. Validation vs. Training Performance Notice how the orange validation loss curve consistently stays below the blue training loss curve, and the orange validation accuracy curve stays above the blue training accuracy curve. This pattern, where validation metrics appear better than training metrics, is often expected when using regularization techniques like Data Augmentation and Dropout. These techniques are applied only during the training phase (model.train()), making that phase slightly harder, but are turned off during validation (model.eval()), allowing the model's full capacity to be assessed on the consistent validation data. Overfitting Check We visually inspect the gap between the training and validation curves. Signs of significant overfitting would include the validation loss (orange) clearly starting to rise while the training loss (blue) continues to fall, or the validation accuracy stalling/dropping while training accuracy keeps climbing. Based on these plots, while there are minor fluctuations, the validation loss remains low and generally trends downwards or flat near the end. The gap between the curves doesn't appear to be dramatically widening, suggesting significant overfitting hasn't set in within these 20 epochs. Optimal Epoch & Training Duration Looking closely at the orange validation loss curve, it appears to reach its minimum value very late in training, around epoch 19 or 20. Similarly, validation accuracy plateaus at its peak in the last few epochs. This suggests that training for the full 20 epochs was beneficial for this specific run and learning rate, and stopping much earlier might have resulted in slightly suboptimal validation performance. TL;DR The plots show stable training with good convergence over 20 epochs. They visually confirm the expected impact of our training-only regularization (Val > Train metrics) and indicate that the model reached its best validation performance near the end of this training run without showing strong signs of overfitting yet. 3. Evaluating Our Pneumonia Detection CNN After training, we need to rigorously evaluate our model to understand its strengths and weaknesses, especially for a medical application like pneumonia detection. Calculating Key Metrics For medical diagnosis tasks, accuracy alone is insufficient. We need to consider Precision Of all cases predicted as pneumonia, how many actually have pneumonia? Recall Of all actual pneumonia cases, how many did we correctly identify? F1-score The harmonic mean of precision and recall Confusion Matrix A table showing true positives, false positives, true negatives, and false negatives Let's implement a detailed evaluation def evaluate_model(model, test_loader, device, class_names) """ Evaluates the model on a given dataloader (e.g., test set). Computes confusion matrix and classification report. """ model.eval() # Set model to evaluation mode all_preds = [] all_labels = [] with torch.no_grad() # Disable gradient calculation for images, labels in test_loader images, labels = images.to(device), labels.to(device) outputs = model(images) _, predictions = torch.max(outputs, 1) all_preds.extend(predictions.cpu().numpy()) all_labels.extend(labels.cpu().numpy()) all_preds = np.array(all_preds) all_labels = np.array(all_labels) # Calculate confusion matrix cm = confusion_matrix(all_labels, all_preds) # Calculate classification report class_report = classification_report( all_labels, all_preds, target_names=class_names, digits=4, zero_division=0 ) # Calculate overall accuracy from the report accuracy = np.trace(cm) / np.sum(cm) # Simple accuracy from confusion matrix return { 'confusion_matrix' cm, 'classification_report' class_report, 'accuracy' accuracy, 'predictions' all_preds, 'true_labels' all_labels } # Evaluate the model eval_results = evaluate_model(model, test_loader, device, class_names) # Print results print("Classification Report") print(eval_results['classification_report']) print(f"Overall Accuracy {eval_results['accuracy'].4f}") You should see output similar to Classification Report precision recall f1-score support NORMAL 0.9780 0.3803 0.5477 234 PNEUMONIA 0.7280 0.9949 0.8407 390 accuracy 0.7644 624 macro avg 0.8530 0.6876 0.6942 624 weighted avg 0.8217 0.7644 0.7308 624 Overall Accuracy 0.7644 Visualizing the Confusion Matrix A confusion matrix provides a clear visual representation of our model's performance def plot_confusion_matrix(confusion_matrix, class_names) plt.figure(figsize=(8, 6)) sns.heatmap( confusion_matrix, annot=True, fmt='d', cmap='Blues', xticklabels=class_names, yticklabels=class_names ) plt.xlabel('Predicted Label') plt.ylabel('True Label') plt.title('Confusion Matrix') plt.tight_layout() plt.show() # Plot confusion matrix plot_confusion_matrix(eval_results['confusion_matrix'], class_names) The confusion matrix shows True Negatives (top-left) Normal X-rays correctly identified as normal False Positives (top-right) Normal X-rays incorrectly identified as pneumonia False Negatives (bottom-left) Pneumonia X-rays incorrectly identified as normal True Positives (bottom-right) Pneumonia X-rays correctly identified as pneumonia Interpreting Results in a Medical Context In medical diagnosis, different types of errors have different consequences False Negatives (missing pneumonia) These are particularly dangerous as a patient with pneumonia might not receive necessary treatment, potentially leading to serious complications. Minimizing these is often a high priority (i.e., maximizing Recall/Sensitivity for the PNEUMONIA class). False Positives (diagnosing pneumonia when it's absent) These may lead to unnecessary treatment, causing stress and potential side effects, but are generally less immediately harmful than false negatives. Minimizing these relates to maximizing Recall/Specificity for the NORMAL class. Examining our actual results from the Classification Report above, we see Pneumonia Detection (Class 1) The model achieves extremely high Recall (Sensitivity) of ~0.9949. This is excellent, meaning it correctly identifies nearly 99.5% of the actual pneumonia cases in the test set, effectively minimizing dangerous False Negatives. However, its Precision for pneumonia is ~0.7280, meaning that when it predicts pneumonia, it's correct only about 73% of the time – the other 27% are False Positives (NORMAL cases misclassified as PNEUMONIA). Normal Case Detection (Class 0) The model still has very low Recall (Specificity) of ~0.3803. This indicates it only correctly identifies about 38% of the actual normal cases; the remaining 62% are misclassified as pneumonia (contributing to the lower precision for the PNEUMONIA class). The Precision for normal cases remains high (~0.9780), meaning if it predicts normal, it's very likely correct, but this model rarely makes that prediction for normal cases. Interpretation These results indicate the model is significantly biased towards predicting PNEUMONIA. It's highly sensitive but lacks specificity. In a real medical scenario The high sensitivity (~99.5%) is valuable for ensuring potential cases aren't missed. The low specificity (~38%) remains highly problematic, likely leading to a large number of unnecessary follow-ups for healthy individuals. While prioritizing sensitivity is common for screening, this level of specificity would likely be impractical. These results strongly suggest that the class imbalance in our training data is heavily influencing the model's predictions. To create a more balanced and clinically useful model, addressing this imbalance directly (using techniques like weighted loss or resampling, as discussed in Section 5) would be the most logical next step. 4. Visualizing Model Predictions Let's visualize some of our model's predictions to better understand its behavior def visualize_predictions(model, dataloader, device, class_names, num_samples=8) """Displays a batch of test images with their true labels and model predictions.""" model.eval() try images, labels = next(iter(dataloader)) except StopIteration print("DataLoader is empty.") return # Ensure we don't request more samples than available in the batch actual_samples = min(num_samples, images.size(0)) if actual_samples <= 0 print("No samples in batch to display.") return images, labels = images[actual_samples], labels[actual_samples] images_device = images.to(device) # Move input data to the correct device # Get model predictions with torch.no_grad() outputs = model(images_device) _, preds = torch.max(outputs, 1) probs = F.softmax(outputs, dim=1) # Move data back to CPU for plotting preds = preds.cpu().numpy() probs = probs.cpu().numpy() images = images.cpu() # Determine subplot layout if actual_samples <= 4 ncols = actual_samples; nrows = 1; figsize = (4 * ncols, 5) else ncols = 4; nrows = 2; figsize = (16, 10) fig, axes = plt.subplots(nrows=nrows, ncols=ncols, figsize=figsize) # Ensure axes is iterable if nrows == 1 and ncols == 1 axes = np.array([axes]) axes = axes.flatten() fig.suptitle("Sample Test Set Predictions", fontsize=16) for i, ax in enumerate(axes) if i < actual_samples img_tensor = images[i] true_label = class_names[labels[i]] pred_label = class_names[preds[i]] confidence = probs[i][preds[i]] # Prepare image for display (C, H, W) -> (H, W, C) img_display = img_tensor.permute(1, 2, 0).numpy() # Display image ax.imshow(img_display.squeeze(), cmap='gray', vmin=0.0, vmax=1.0) # Set title with prediction info and color coding title_color = 'green' if pred_label == true_label else 'red' title = f"True {true_label}Pred {pred_label}Conf {confidence.2f}" ax.set_title(title, color=title_color) ax.axis('off') else ax.axis('off') plt.tight_layout() plt.subplots_adjust(top=0.92) # Adjust layout for suptitle plt.show() # Visualize model predictions on the test set print("Visualizing sample predictions from the test set...") # Create a TEMPORARY DataLoader with shuffling enabled JUST for visualization # This helps ensure we see a mix of classes in the first batch we grab. # The 'test_loader' used for actual evaluation remains unshuffled. temp_vis_loader = DataLoader( dataset=test_dataset, # Use the same test_dataset batch_size=batch_size, # Use the same batch size shuffle=True # Shuffle ON for this temporary loader ) visualize_predictions(model, temp_vis_loader, device, class_names) This visualization provides concrete examples of the model's behavior on the test set We can clearly see examples of both correct (green titles) and incorrect (red titles) predictions made by the model. It allows us to observe the model's confidence for each prediction. Notice in this batch that the confidence scores are generally quite high (often >0.80), even for some of the incorrect classifications. Most importantly, we can identify potential patterns in the errors. In this specific sample batch, the errors primarily consist of True NORMAL images being incorrectly classified as PNEUMONIA, sometimes with high confidence. This visually reinforces the low Specificity (low Recall for the NORMAL class) identified in our quantitative evaluation metrics and highlights the model's tendency to misclassify normal cases. Making Predictions on Random Images Evaluating metrics like precision and recall gives us an overall sense of performance, but looking at individual predictions can provide more intuition. Let's see how our trained model performs on a randomly selected individual X-ray image from the NORMAL class in the test set. First, here's the helper function we'll use to load, preprocess, and get a prediction for a single image path def predict_image(model, image_path, transform, device, class_names) """Loads a single image, preprocesses it, and returns model prediction details.""" try # Load the image using PIL image = Image.open(image_path) except FileNotFoundError print(f"Error Image file not found at {image_path}") return None except Exception as e print(f"Error opening image {image_path} {e}") return None # Preprocess Apply validation/test transforms, add batch dimension, move to device image_tensor = transform(image).unsqueeze(0).to(device) # Make prediction model.eval() # Ensure model is in evaluation mode with torch.no_grad() # Disable gradient calculations output = model(image_tensor) # Output raw logits probabilities = F.softmax(output, dim=1) # Probabilities # Get the highest probability score and the corresponding class index confidence, predicted_class_idx = torch.max(probabilities, 1) # Extract results class_idx = predicted_class_idx.item() class_name = class_names[class_idx] # Map index to class name confidence_score = confidence.item() # Return results as a dictionary return { 'class_id' class_idx, 'class_name' class_name, 'confidence' confidence_score, 'probabilities' probabilities[0].cpu().numpy() # All class probabilities } Now, let's use this function on a random image from the test/NORMAL directory try normal_dir = os.path.join(test_dir, "NORMAL") # Target the NORMAL directory # Get only .jpeg files from the directory normal_test_files = [f for f in os.listdir(normal_dir) if f.lower().endswith('.jpeg')] if not normal_test_files print(f"No NORMAL test images found in {normal_dir}.") else # Select a random image file random_filename = random.choice(normal_test_files) test_image_path = os.path.join(normal_dir, random_filename) print(f"Predicting on random NORMAL image {random_filename}") # Get prediction using the function result = predict_image(model, test_image_path, val_test_transforms, device, class_names) if result # Display the prediction details print(f" Actual class NORMAL") # State the true class print(f" Predicted class {result['class_name']}") print(f" Confidence {result['confidence'].4f}") print(f" Class probabilities Normal={result['probabilities'][0].4f}, Pneumonia={result['probabilities'][1].4f}") # Visualize the image with prediction try img = Image.open(test_image_path) plt.figure(figsize=(6, 6)) plt.imshow(img, cmap='gray') # Include TRUE label in title for clarity plt.title(f"True NORMAL | Prediction {result['class_name']} ({result['confidence'].4f})") plt.axis('off') plt.show() except Exception as e print(f"Error displaying image {test_image_path} {e}") except FileNotFoundError print(f"Error Directory {normal_dir} not found.") except Exception as e print(f"An error occurred during prediction example {e}") Predicting on random NORMAL image IM-0011-0001-0001.jpeg Actual class NORMAL Predicted class PNEUMONIA Confidence 0.7053 Class probabilities Normal=0.2947, Pneumonia=0.7053 Here, we took a random test X-ray image (IM-0011-0001-0001.jpeg) known to be NORMAL. Our model, however, incorrectly predicted it as PNEUMONIA with moderate confidence (approx. 70.5%). This specific misclassification provides a clear example of the main weakness identified in our evaluation metrics the model's difficulty in correctly recognizing NORMAL cases (achieving only ~38.0% recall/specificity according to the Classification Report). Errors like this, where NORMAL images are falsely predicted as PNEUMONIA (False Positives), are why the overall Precision for the PNEUMONIA class was limited to ~72.8%. ****When the model predicts PNEUMONIA, roughly 27% of those predictions are actually NORMAL cases being misclassified. While the model remains excellent at catching actual PNEUMONIA (with ~99.5% recall/sensitivity), this tendency to misclassify NORMAL images highlights the impact of the class imbalance. Looking at the specific image, we can see prominent normal structures (bronchial/vascular markings); it's plausible that the model, biased by the imbalance, struggles to differentiate these complex normal patterns from potential abnormalities. Addressing this bias to improve specificity would clearly improve the model's clinical utility. This leads us nicely into exploring common training issues and techniques to mitigate them. 5. Addressing Common CNN Training Issues Now that we've trained and evaluated our model, we've seen some promising results but also potential areas for improvement (like the low specificity driven by class imbalance). Let's explore common issues encountered during CNN development and strategies to address them, considering our specific pneumonia detection task. Diagnosing and Addressing Overfitting Overfitting occurs when a model learns the training data too well, including its noise and specific quirks, rather than the underlying general patterns. This leads to poor performance on new, unseen data. Signs of overfitting in our training plots would include Training accuracy becoming much higher than validation accuracy. Training loss continuing to decrease significantly while validation loss plateaus or starts increasing. Strategy Early Stopping If you observe validation loss starting to increase (a clear sign of overfitting), one effective strategy is early stopping. Concept Monitor the validation loss after each epoch. Save the model's state whenever the validation loss reaches a new minimum. If the validation loss fails to improve for a predefined number of epochs (e.g., 5 or 10, known as "patience"), stop the training process. Finally, load the saved model state that achieved the best validation loss. Example Application For our project, this would involve modifying the training loop to keep track of the best epoch_val_loss seen so far, saving model.state_dict() at that point, and halting if the loss doesn't improve for the specified patience period. Handling Underfitting Underfitting is the opposite problem the model fails to learn the training data well enough, resulting in poor performance on both the training and validation/test sets. This often suggests the model is too simple or hasn't trained sufficiently. Potential Strategies Increase Model Complexity Make the model more powerful so it can capture more complex patterns. Example Application We could add a fourth convolutional block to our PneumoniaCNN definition or increase the number of output channels in the existing nn.Conv2d layers (e.g., going from 32 -> 64 -> 128 to perhaps 64 -> 128 -> 256). Train Longer Give the model more time to learn by increasing the number of training epochs. Example Application Simply call our train_model function with a larger value, like num_epochs=30 or num_epochs=50, while carefully monitoring for signs of overfitting using the validation metrics. Reduce Regularization Techniques like dropout prevent overfitting but can hinder learning if applied too aggressively when the model is underfitting. Example Application We could try lowering the dropout probability in our fully connected layers, for instance, changing nn.Dropout(p=0.5) to nn.Dropout(p=0.3). If using weight decay in the optimizer, we might reduce its strength. Learning Rate Adjustment The learning rate might be too low (slow learning) or too high (preventing convergence). Experimenting or using a scheduler can help. Example Application We could try initializing the Adam optimizer with a slightly different learning rate, like lr=0.001 or lr=0.005. Alternatively, we could implement a learning rate scheduler (e.g., torch.optim.lr_scheduler.ReduceLROnPlateau) that automatically reduces the learning rate if the validation loss stagnates. Addressing Class Imbalance As our verification step showed, the training set has a roughly 31 ratio of PNEUMONIA to NORMAL samples. This imbalance likely contributed to our model's bias towards predicting PNEUMONIA (high sensitivity, low specificity). Common strategies include Weighted Loss Function Modify the loss calculation to penalize errors on the minority class (NORMAL) more heavily. Example Application Calculate weights inversely proportional to class frequency (e.g., assign a weight of ~3 to the NORMAL class and ~1 to the PNEUMONIA class) and pass these weights to the weight parameter of nn.CrossEntropyLoss when defining our criterion. Resampling Adjust the sampling process during training to create more balanced batches. Example Application Oversampling the minority class involves drawing more samples (with replacement) from the NORMAL images during each epoch, perhaps using PyTorch's WeightedRandomSampler with the DataLoader. Undersampling the majority class involves randomly discarding some PNEUMONIA samples to match the number of NORMAL samples, though this risks losing potentially useful information. Generate Synthetic Data Create artificial examples of the minority class. Example Application This often involves more advanced techniques like SMOTE (Synthetic Minority Over-sampling Technique) or using Generative Adversarial Networks (GANs) to create new, realistic-looking NORMAL X-ray images, though implementing these is beyond the scope of this tutorial. Choosing the right strategy often involves experimentation. For class imbalance, using a weighted loss or resampling via WeightedRandomSampler are often effective starting points. Review and Next Steps Congratulations! You've successfully navigated this two-part tutorial series, journeying from the fundamentals of Convolutional Neural Networks all the way to building, training, and evaluating a practical pneumonia detection model using PyTorch. You've seen the entire workflow, from defining the architecture in Part 1 to preparing data, implementing training loops, interpreting results, and considering common challenges in Part 2. What You've Learned As we wrap up, let's distill the most important concepts and practices covered across both tutorials End-to-End Workflow Building an effective computer vision solution involves a complete pipeline careful data preparation (verification, splitting, augmentation, transformation), thoughtful model architecture definition (often using PyTorch's object-oriented nn.Module structure), implementing robust training loops (managing device placement and model modes), and performing rigorous evaluation tailored to the problem. Data is Foundational The quality and handling of your data are paramount. Accurate verification, appropriate splitting (train/validation/test), deliberate preprocessing choices (like resizing or grayscale conversion), and techniques like data augmentation significantly impact model performance and reliability. Evaluate Beyond Accuracy Especially for real-world applications like medical diagnosis, relying solely on accuracy can be misleading, particularly with imbalanced datasets. Metrics like precision, recall (sensitivity/specificity), F1-score, and confusion matrices provide a much deeper understanding of model strengths and weaknesses for each class. Practical Training Details Matter Correctly switching between model.train() and model.eval() is essential for layers like Dropout and BatchNorm to function properly. Being aware of potential issues like overfitting or class imbalance and knowing strategies to address them (e.g., early stopping, learning rate scheduling, weighted loss, resampling) are key practical skills for refining models. What You Can Try Next Your journey into computer vision with PyTorch doesn't have to end here! To deepen your skills, consider exploring these areas Transfer Learning Instead of training from scratch, leverage powerful models (like ResNet, VGG, DenseNet) pre-trained on large datasets (like ImageNet) and fine-tune them for your specific task. This often leads to better performance with less data and faster training. Cross-Validation Implement k-fold cross-validation for a more robust evaluation of your model's performance, reducing the dependency on a single train-validation split. Hyperparameter Tuning Systematically experiment with different learning rates, batch sizes, optimizer choices, network architectures, or augmentation strategies. Explainability Use techniques like Grad-CAM, SHAP, or LIME to understand why your model makes certain predictions. Visualize the image regions that most influence its decision. This is important for building trust, especially in medical AI. Remember that deep learning is as much an art as it is a science—experimentation, careful analysis, and domain knowledge all play important roles in creating effective solutions. Keep practicing these skills, and you'll be well-equipped to solve real-world problems with computer vision in PyTorch. Additional Resources To continue your learning journey PyTorch Documentation - Comprehensive reference for all PyTorch functions Sequence Models in PyTorch - If you're interested in extending your skills to sequential data Natural Language Processing with PyTorch - For processing text data with PyTorch Medical Image Analysis Papers - Recent research in medical image classification

.NET 8 Performance Improvements in .NET MAUI

The major focus for .NET MAUI in the .NET 8 release is quality. As such, alot of our focus has been fixing bugs instead of chasing lofty performance goals. In .NET 8, we merged 1,559 pull requests that closed 596 total issues. These include changes from the .NET MAUI team as well as the .NET MAUI community. We are optimistic that this should result in a significant increase in quality in .NET 8. However! We still have plenty of performance changes to showcase. Building upon the fundamental performance improvements in .NET 8 we discover “low-hanging” fruit constantly, and there were high-voted performance issues on GitHub we tried to tackle. Our goal is to continue to make .NET MAUI faster in each release, read on for details! For a review of the performance improvements in past releases, see our posts for .NET 6 and 7. This also gives you an idea of the improvements you would see migrating from Xamarin.Forms to .NET MAUI .NET 7 Performance Improvements in .NET MAUI .NET 6 Performance Improvements in .NET MAUI Table Of Contents New features AndroidStripILAfterAOT AndroidEnableMarshalMethods NativeAOT on iOS Build & Inner Loop Performance Filter Android ps -A output with grep Port WindowsAppSDK usage of vcmeta.dll to C# Improvements to remote iOS builds on Windows Improvements to Android inner-loop XAML Compilation no longer uses LoadInSeparateAppDomain Performance or App Size Improvements Structs and IEquatable in .NET MAUI Fix performance issue in {AppThemeBinding} Address CA1307 and CA1309 for performance Address CA1311 for performance Remove unused ViewAttachedToWindow event on Android Remove unneeded System.Reflection for {Binding} Use StringComparer.Ordinal for Dictionary and HashSet Reduce Java interop in MauiDrawable on Android Improve layout performance of Label on Android Reduce Java interop calls for controls in .NET MAUI Improve performance of Entry.MaxLength on Android Improve memory usage of CollectionView on Windows Use UnmanagedCallersOnlyAttribute on Apple platforms Faster Java interop for strings on Android Faster Java interop for C# events on Android Use Function Pointers for JNI Removed Xamarin.AndroidX.Legacy.Support.V4 Deduplication of generics on iOS and macOS Fix System.Linq.Expressions implementation on iOS-like platforms Set DynamicCodeSupport=false for iOS and Catalyst Memory Leaks Memory Leaks and Quality Diagnosing leaks in .NET MAUI Patterns that cause leaks C# events Circular references on Apple platforms Roslyn analyzer for Apple platforms Tooling and Documentation Simplified dotnet-trace and dotnet-dsrouter dotnet-gcdump Support for Mobile New Features AndroidStripILAfterAOT Once Upon A Time we had a brilliant thought if AOT pre-compiles C# methods, do we need the managed method anymore? Removing the C# method body would allow assemblies to be smaller. .NET iOS applications already do this, so why not Android as well? While the idea is straightforward, implementation was not iOS uses “Full” AOT, which AOT’s all methods into a form that doesn’t require a runtime JIT. This allowed iOS to run cil-strip, removing all method bodies from all managed types. At the time, Xamarin.Android only supported “normal” AOT, and normal AOT requires a JIT for certain constructs such as generic types and generic methods. This meant that attempting to run cil-strip would result in runtime errors if a method body was removed that was actually required at runtime. This was particularly bad because cil-strip could only remove all method bodies! We are re-intoducing IL stripping for .NET 8. Add a new $(AndroidStripILAfterAOT) MSBuild property. When true, the <MonoAOTCompiler/> task will track which method bodies were actually AOT’d, storing this information into %(_MonoAOTCompiledAssemblies.MethodTokenFile), and the new <ILStrip/> task will update the input assemblies, removing all method bodies that can be removed. By default enabling $(AndroidStripILAfterAOT) will override the default $(AndroidEnableProfiledAot) setting, allowing all trimmable AOT’d methods to be removed. This choice was made because $(AndroidStripILAfterAOT) is most useful when AOT-compiling your entire application. Profiled AOT and IL stripping can be used together by explicitly setting both within the .csproj, but with the only benefit being a small .apk size improvement <PropertyGroup Condition=" '$(Configuration)' == 'Release' "> <AndroidStripILAfterAOT>true</AndroidStripILAfterAOT> <AndroidEnableProfiledAot>true</AndroidEnableProfiledAot> </PropertyGroup> .apk size results for a dotnet new android app $(AndroidStripILAfterAOT) $(AndroidEnableProfiledAot) .apk size true true 7.7MB true false 8.1MB false true 7.7MB false false 8.4MB Note that AndroidStripILAfterAOT=false and AndroidEnableProfiledAot=true is the default Release configuration environment, for 7.7MB. A project that only sets AndroidStripILAfterAOT=true implicitly sets AndroidEnableProfiledAot=false, resulting in an 8.1MB app. See xamarin-android#8172 and dotnet/runtime#86722 for details about this feature. AndroidEnableMarshalMethods .NET 8 introduces a new experimental setting for Release configurations <PropertyGroup Condition=" '$(Configuration)' == 'Release' "> <AndroidEnableMarshalMethods>true</AndroidEnableMarshalMethods>  <RuntimeIdentifier>android-arm64</RuntimeIdentifier> </PropertyGroup> We hope to enable this feature by default in .NET 9, but for now we are providing the setting as an opt-in, experimental feature. Applications that only target one architecture, such as RuntimeIdentifier=android-arm64, will likely be able to enable this feature without issue. Background on Marshal Methods A JNI marshal method is a JNI-callable function pointer provided to JNIEnvRegisterNatives(). Currently, JNI marshal methods are provided via the interaction between code we generate and JNINativeWrapper.CreateDelegate() Our code-generator emits the “actual” JNI-callable method. JNINativeWrapper.CreateDelegate() uses System.Reflection.Emit to wrap the method for exception marshaling. JNI marshal methods are needed for all Java-to-C# transitions. Consider the virtual Activity.OnCreate() method partial class Activity { static Delegate? cb_onCreate_Landroid_os_Bundle_; static Delegate GetOnCreate_Landroid_os_Bundle_Handler () { if (cb_onCreate_Landroid_os_Bundle_ == null) cb_onCreate_Landroid_os_Bundle_ = JNINativeWrapper.CreateDelegate ((_JniMarshal_PPL_V) n_OnCreate_Landroid_os_Bundle_); return cb_onCreate_Landroid_os_Bundle_; } static void n_OnCreate_Landroid_os_Bundle_ (IntPtr jnienv, IntPtr native__this, IntPtr native_savedInstanceState) { var __this = globalJava.Lang.Object.GetObject<Android.App.Activity> (jnienv, native__this, JniHandleOwnership.DoNotTransfer)!; var savedInstanceState = globalJava.Lang.Object.GetObject<Android.OS.Bundle> (native_savedInstanceState, JniHandleOwnership.DoNotTransfer); __this.OnCreate (savedInstanceState); } // Metadata.xml XPath method reference path="/api/package[@name='android.app']/class[@name='Activity']/method[@name='onCreate' and count(parameter)=1 and parameter[1][@type='android.os.Bundle']]" [Register ("onCreate", "(Landroid/os/Bundle;)V", "GetOnCreate_Landroid_os_Bundle_Handler")] protected virtual unsafe void OnCreate (Android.OS.Bundle? savedInstanceState) => ... } Activity.n_OnCreate_Landroid_os_Bundle_() is the JNI marshal method, responsible for marshaling parameters from JNI values into C# types, forwarding the method invocation to Activity.OnCreate(), and (if necessary) marshaling the return value back to JNI. Activity.GetOnCreate_Landroid_os_Bundle_Handler() is part of the type registration infrastructure, providing a Delegate instance to RegisterNativeMembers .RegisterNativeMembers(), which is eventually passed to JNIEnvRegisterNatives(). While this works, it’s not incredibly performant unless using one of the optimized delegate types added in xamarin-android#6657, System.Reflection.Emit is used to create a wrapper around the marshal method, which is something we’ve wanted to avoid doing for years. Thus, the idea since we’re already bundling a native toolchain and using LLVM-IR to produce libxamarin-app.so, what if we emitted Java native method names and skipped all the done as part of Runtime.register() and JNIEnv.RegisterJniNatives()? Given class MyActivity Activity { protected override void OnCreate(Bundle? state) => ... } During the build, libxamarin-app.so would contain the function JNIEXPORT void JNICALL Java_crc..._MyActivity_n_1onCreate (JNIEnv *env, jobject self, jobject state); During App runtime, the Runtime.register() invocation present in Java Callable Wrappers would either be omitted or would be a no-op, and Android/JNI would instead resolve MyActivity.n_onCreate() as Java_crc..._MyActivity_n_1onCreate(). We call this effort “LLVM Marshal Methods”, which is currently experimental in .NET 8. Many of the specifics are still being investigated, and this feature will be spread across various areas. See xamarin-android#7351 for details about this experimental feature. NativeAOT on iOS In .NET 7, we started an experiment to see what it would take to support NativeAOT on iOS. Going from prototype to an initial implementation .NET 8 Preview 6 included NativeAOT as an experimental feature for iOS. To opt into NativeAOT in a MAUI iOS project, use the following settings in your project file <PropertyGroup Condition="$([MSBuild]GetTargetPlatformIdentifier('$(TargetFramework)')) == 'ios' and '$(Configuration)' == 'Release'">  <PublishAot>true</PublishAot> </PropertyGroup> Then to build the application for an iOS device $ dotnet publish -f net8.0-ios -r ios-arm64 MSBuild version 17.8.0+6cdef4241 for .NET ... Build succeeded. 0 Error(s) Note We may consider unifying and improving MSBuild property names for this feature in future .NET releases. To do a one-off build at the command-line you may also need to specify -pPublishAotUsingRuntimePack=true in addition to -pPublishAot=true. One of the main culprits for the first release was how the iOS workload supports Objective-C interoperability. The problem was mainly related to the type registration system which is the key component for efficiently supporting iOS-like platforms (see docs for details). In its implementation, the type registration system depends on type metadata tokens which are not available with NativeAOT. Therefore, in order to leverage the benefits of highly efficient NativeAOT runtime, we had to adapt. dotnet/runtime#80912 includes the discussion around how to tackle this problem, and finally in xamarin-macios#18268 we implemented a new managed static registrar that works with NativeAOT. The new managed static registrar does not just benefit us with being compatible with NativeAOT, but is also much faster than the default one, and is available for all supported runtimes (see docs for details). Along the way, we had a great help from our GH community and their contribution (code reviews, PRs) was essential to helps us move forward quickly and deliver this feature on time. A few from many PR’s that helped and unblocked us on our journey were dotnet/runtime#77956 dotnet/runtime#78280 dotnet/runtime#82317 dotnet/runtime#85996 and the list goes on… As .NET 8 Preview 6 came along, we finally managed to release our first version of the NativeAOT on iOS which also supports MAUI. See the blog post on .NET 8 Preview 6 for details about what we were able to accomplish in the initial release. In subsequent .NET 8 releases, results improved quite a bit, as we were identifying and resolving issues along the way. The graph below shows the .NET MAUI iOS template app size comparison throughout the preview releases We had steady progress and estimated size savings reported, due to fixing the following issues dotnet/runtime#87924 – fixed major NativeAOT size issue with AOT-incompatible code paths in System.Linq.Expressions and also made fully NativeAOT compatible when targeting iOS xamarin-macios#18332 – reduced the size of __LINKEDIT Export Info section in stripped binaries Furthermore, in the latest RC 1 release the app size went even further down reaching -50% smaller apps for the template .NET MAUI iOS applications compared to Mono. Most impactful issues/PRs that contributed to this xamarin-macios#18734 – Make Full the default link mode for NativeAOT xamarin-macios#18584 – Make the codebase trimming compatible through a series of PRs. Even though app size was our primary metric to focus on, for the RC 1 release, we also measured startup time performance comparisons for a .NET MAUI iOS template app comparing NativeAOT and Mono where NativeAOT results with almost 2x faster startup time. Key Takeaways For NativeAOT scenarios on iOS, changing the default link mode to Full (xamarin-macios#18734) is probably the biggest improvement for application size. But at the same time, this change can also break applications which are not fully AOT and trim-compatible. In Full link mode, the trimmer might trim away AOT incompatible code paths (think about reflection usage) which are accessed dynamically at runtime. Full link mode is not the default configuration when using the Mono runtime, so it is possible that some applications are not fully AOT-compatible. Supporting NativeAOT on iOS is an experimental feature and still a work-in-progress, and our plan is to address the potential issues with Full link mode incrementally As a first step, we enabled trim, AOT, and single-file warnings by default in xamarin-macios#18571. The enabled warnings should make our customers aware at build-time, whether a use of a certain framework or a library, or some C# constructs in their code, is incompatible with NativeAOT – and could crash at runtime. This information should guide our customers to write AOT-compatible code, but also to help us improve our frameworks and libraries with the same goal of fully utilising the benefits of AOT compilation. The second step, was clearing up all the warnings coming from Microsoft.iOS and System.Private.CoreLib assemblies reported for a template iOS application with xamarin-macios#18629 and dotnet/runtime#91520. In future releases, we plan to address the warnings coming from the MAUI framework and further improve the overall user-experience. Our goal is to have fully AOT and trim-compatible frameworks. .NET 8 will support targeting iOS platforms with NativeAOT as an opt-in feature and shows great potential by generating up to 50% smaller and 50% faster startup compared to Mono. Considering the great performance that NativeAOT promises, please help us on this journey and try out your applications with NativeAOT and report any potential issues. At the same time, let us know when NativeAOT “just works” out-of-the-box. To follow future progress, see dotnet/runtime#80905. Last but not least, we would like to thank our GH contributors, who are helping us make NativeAOT on iOS possible. Build & Inner Loop Performance Filter Android ps -A output with grep When profiling the Android inner loop for a .NET MAUI project with PerfView we found around 1.2% of CPU time was spent just trying to get the process ID of the running Android application. When changing Tools > Options > Xamarin > Xamarin Diagnostics output verbosity to be Diagnostics, you could see -- Start GetProcessId - 12/02/2022 110557 (96.9929ms) -- [INPUT] ps -A [OUTPUT] USER PID PPID VSZ RSS WCHAN ADDR S NAME root 1 0 10943736 4288 0 0 S init root 2 0 0 0 0 0 S [kthreadd] ... Hundreds of more lines! u0_a993 14500 1340 14910808 250404 0 0 R com.companyname.mauiapp42 -- End GetProcessId -- The Xamarin/.NET MAUI extension in Visual Studio polls every second to see if the application has exited. This is useful for changing the play/stop button state if you force close the app, etc. Testing on a Pixel 5, we could see the command is actually 762 lines of output! > (adb shell ps -A).Count 762 What we could do instead is something like > adb shell "ps -A | grep -w -E 'PID|com.companyname.mauiapp42'" Where we pipe the output of ps -A to the grep command on the Android device. Yes, Android has a subset of unix commands available! We filter on either a line containing PID or your application’s package name. The result is now the IDE is only parsing 4 lines [INPUT] ps -A | grep -w -E 'PID|com.companyname.mauiapp42' [OUTPUT] USER PID PPID VSZ RSS WCHAN ADDR S NAME u0_a993 12856 1340 15020476 272724 0 0 S com.companyname.mauiapp42 This not only improves memory used to split and parse this information in C#, but adb is also transmitting way less bytes across your USB cable or virtually from an emulator. This feature shipped in recent versions of Visual Studio 2022, improving this scenario for all Xamarin and .NET MAUI customers. Port WindowsAppSDK usage of vcmeta.dll to C# We found that every incremental build of a .NET MAUI project running on Windows spent time in Top 10 most expensive tasks CompileXaml = 3.972 s ... various tasks ... This is the XAML compiler for WindowsAppSDK, that compiles the WinUI3 flavor of XAML (not .NET MAUI XAML). There is very little XAML of this type in .NET MAUI projects, in fact, the only file is Platforms/Windows/App.xaml in the project template. Interestingly, if you installed the Desktop development with C++ workload in the Visual Studio installer, this time just completely went away! Top 10 most expensive tasks ... various tasks ... CompileXaml = 9 ms The WindowsAppSDK XAML compiler p/invokes into a native library from the C++ workload, vcmeta.dll, to calculate a hash for .NET assembly files. This is used to make incremental builds fast — if the hash changes, compile the XAML again. If vcmeta.dll was not found on disk, the XAML compiler was effectively “recompiling everything” on every incremental build. For an initial fix, we simply included a small part of the C++ workload as a dependency of .NET MAUI in Visual Studio. The slightly larger install size was a good tradeoff for saving upwards of 4 seconds in incremental build time. Next, we implemented vcmeta.dll‘s hashing functionality in plain C# with System.Reflection.Metadata to compute indentical hash values as before. Not only was this implementation better, in that we could drop a dependency on the C++ workload, but it was also faster! The time to compute a single hash Method Mean Error StdDev Native 217.31 us 1.704 us 1.594 us Managed 86.43 us 1.700 us 2.210 us Some of the reasons this was faster No p/invoke or COM-interfaces involved. System.Reflection.Metadata has a fast struct-based API, perfect for iterating over types in a .NET assembly and computing a hash value. The end result being that CompileXaml might actually be even faster than 9ms in incremental builds. This feature shipped in WindowsAppSDK 1.3, which is now used by .NET MAUI in .NET 8. See WindowsAppSDK#3128 for details about this improvement. Improvements to remote iOS builds on Windows Comparing inner loop performance for iOS, there was a considerable gap between doing “remote iOS” development on Windows versus doing everything locally on macOS. Many small improvements were made, based on comparing inner-loop .binlog files recorded on macOS versus one recorded inside Visual Studio on Windows. Some examples include maui#12747 don’t explicitly copy files to the build server xamarin-macios#16752 do not copy files to build server for a Delete operation xamarin-macios#16929 batch file deletion via DeleteFilesAsync xamarin-macios#17033 cache AOT compiler path Xamarin/MAUI Visual Studio extension when running dotnet-install.sh on remote build hosts, set the explicit processor flag for M1 Macs. We also made some improvements for all iOS & MacCatalyst projects, such as xamarin-macios#16416 don’t process assemblies over and over again Improvements to Android inner-loop We also made many small improvements to the “inner-loop” on Android — most of which were focused in a specific area. Previously, Xamarin.Forms projects had the luxury of being organized into multiple projects, such as YourApp.Android.csproj Xamarin.Android application project YourApp.iOS.csproj Xamarin.iOS application project YourApp.csproj netstandard2.0 class library Where almost all of the logic for a Xamarin.Forms app was contained in the netstandard2.0 project. Nearly all the incremental builds would be changes to XAML or C# in the class library. This structure enabled the Xamarin.Android MSBuild targets to completely skip many Android-specific MSBuild steps. In .NET MAUI, the “single project” feature means that every incremental build has to run these Android-specific build steps. In focusing specifically improving this area, we made many small changes, such as java-interop#1061 avoid string.Format() java-interop#1064 improve ToJniNameFromAttributesForAndroid java-interop#1065 avoid File.Exists() checks java-interop#1069 fix more places to use TypeDefinitionCache java-interop#1072 use less System.Linq for custom attributes java-interop#1103 use MemoryMappedFile when using Mono.Cecil xamarin-android#7621 avoid File.Exists() checks xamarin-android#7626 perf improvements for LlvmIrGenerator xamarin-android#7652 fast path for <CheckClientHandlerType/> xamarin-android#7653 delay ToJniName when generating AndroidManifest.xml xamarin-android#7686 lazily populate Resource lookup These changes should improve incremental builds in all .NET 8 Android project types. XAML Compilation no longer uses LoadInSeparateAppDomain Looking at the JITStats report in PerfView (for MSBuild.exe) Name JitTime (ms) Microsoft.Maui.Controls.Build.Tasks.dll 214.0 Mono.Cecil 119.0 It appears that Microsoft.Maui.Controls.Build.Tasks.dll was spending a lot of time in the JIT. What was confusing, is this was an incremental build where everything should already be loaded. The JIT’s work should be done already? The cause appears to be usage of the [LoadInSeparateAppDomain] attribute defined by the <XamlCTask/> in .NET MAUI. This is an MSBuild feature that gives MSBuild tasks to run in an isolated AppDomain — with an obvious performance drawback. However, we couldn’t just remove it as there would be complications… [LoadInSeparateAppDomain] also conveniently resets all static state when <XamlCTask/> runs again. Meaning that future incremental builds would potentially use old (garbage) values. There are several places that cache Mono.Cecil objects for performance reasons. Really weird bugs would result if we didn’t address this. So, to actually make this change, we reworked all static state in the XAML compiler to be stored in instance fields & properties instead. This is a general software design improvement, in addition to giving us the ability to safely remove [LoadInSeparateAppDomain]. The results of this change, for an incremental build on a Windows PC Before XamlCTask = 743 ms XamlCTask = 706 ms XamlCTask = 692 ms After XamlCTask = 128 ms XamlCTask = 134 ms XamlCTask = 117 ms This saved about ~587ms on incremental builds on all platforms, an 82% improvement. This will help even more on large solutions with multiple .NET MAUI projects, where <XamlCTask/> runs multiple times. See maui#11982 for further details about this improvement. Performance or App Size Improvements Structs and IEquatable in .NET MAUI Using the Visual Studio’s .NET Object Allocation Tracking profiler on a customer .NET MAUI sample application, we saw Microsoft.Maui.WeakEventManager+Subscription Allocations 686,114 Bytes 21,955,648 This seemed like an exorbitant amount of memory to be used in a sample application’s startup! Drilling in to see where these struct‘s were being created System.Collections.Generic.ObjectEqualityComparer<Microsoft.Maui.WeakEventManager+Subscription>.IndexOf() The underlying problem was this struct didn’t implement IEquatable<T> and was being used as the key for a dictionary. The CA1815 code analysis rule was designed to catch this problem. This is not a rule that is enabled by default, so projects must opt into it. To solve this Subscription is internal to .NET MAUI, and its usage made it possible to be a readonly struct. This was just an extra improvement. We made CA1815 a build error across the entire dotnet/maui repository. We implemented IEquatable<T> for all struct types. After these changes, we could no longer found Microsoft.Maui.WeakEventManager+Subscription in memory snapshots at all. Which saved ~21 MB of allocations in this sample application. If your own projects have usage of struct, it seems quite worthwhile to make CA1815 a build error. A smaller, targeted version of this change was backported to MAUI in .NET 7. See maui#13232 for details about this improvement. Fix performance issue in {AppThemeBinding} Profiling a .NET MAUI sample application from a customer, we noticed a lot of time spent in {AppThemeBinding} and WeakEventManager while scrolling 2.08s (17%) microsoft.maui.controls!Microsoft.Maui.Controls.AppThemeBinding.Apply(object,Microsoft.Maui.Controls.BindableObject,Micr... 2.05s (16%) microsoft.maui.controls!Microsoft.Maui.Controls.AppThemeBinding.AttachEvents() 2.04s (16%) microsoft.maui!Microsoft.Maui.WeakEventManager.RemoveEventHandler(System.EventHandler`1<TEventArgs_REF>,string) The following was happening in this application The standard .NET MAUI project template has lots of {AppThemeBinding} in the default Styles.xaml. This supports Light vs Dark theming. {AppThemeBinding} subscribes to Application.RequestedThemeChanged So, every MAUI view subscribe to this event — potentially multiple times. Subscribers are a Dictionary<string, List<Subscriber>>, where there is a dictionary lookup followed by a O(N) search for unsubscribe operations. There is potentially a usecase here to come up with a generalized “weak event” pattern for .NET. The implementation currently in .NET MAUI came over from Xamarin.Forms, but a generalized pattern could be useful for .NET developers using other UI frameworks. To make this scenario fast, for now, in .NET 8 Before For any {AppThemeBinding}, it calls both RequestedThemeChanged -= OnRequestedThemeChanged O(N) time RequestedThemeChanged += OnRequestedThemeChanged constant time Where the -= is notably slower, due to possibly 100s of subscribers. After Create an _attached boolean, so we know know the “state” if it is attached or not. New bindings only call +=, where -= will now only be called by {AppThemeBinding} in rare cases. Most .NET MAUI apps do not “unapply” bindings, but -= would only be used in that case. See the full details about this fix in maui#14625. See dotnet/runtime#61517 for how we could implement “weak events” in .NET in the future. Address CA1307 and CA1309 for performance Profiling a .NET MAUI sample application from a customer, we noticed time spent during “culture-aware” string operations 77.22ms microsoft.maui!Microsoft.Maui.Graphics.MauiDrawable.SetDefaultBackgroundColor() 42.55ms System.Private.CoreLib!System.String.ToLower() This case, we can improve by simply calling ToLowerInvariant() instead. In some cases you might even consider using string.Equals() with StringComparer.Ordinal. In this case, our code was further reviewed and optimized in Reduce Java interop in MauiDrawable on Android. In .NET 7, we added CA1307 and CA1309 code analysis rules to catch cases like this, but it appears we missed some in Microsoft.Maui.Graphics.dll. These are likely useful rules to enable in your own .NET MAUI applications, as avoiding all culture-aware string operations can be quite impactful on mobile. See maui#14627 for details about this improvement. Address CA1311 for performance After addressing the CA1307 and CA1309 code analysis rules, we took things further and addressed CA1311. As mentioned in the turkish example, doing something like string text = something.ToUpper(); switch (text) { ... } Can actually cause unexpected behavior in Turkish locales, because in Turkish, the character I (Unicode 0049) is considered the upper case version of a different character ý (Unicode 0131), and i (Unicode 0069) is considered the lower case version of yet another character Ý (Unicode 0130). ToLowerInvariant() and ToUpperInvariant() are also better for performance as an invariant ToLower / ToUpper operation is slightly faster. Doing this also avoids loading the current culture, improving startup performance. There are cases where you would want the current culture, such as in a CaseConverter type in .NET MAUI. To do this, you simply have to be explicit in which culture you want to use return ConvertToUpper ? v.ToUpper(CultureInfo.CurrentCulture) v.ToLower(CultureInfo.CurrentCulture); The goal of this CaseConverter is to display upper or lowercase text to a user. So it makes sense to use the CurrentCulture for this. See maui#14773 for details about this improvement. Remove unused ViewAttachedToWindow event on Android Every Label in .NET MAUI was subscribing to public class MauiTextView AppCompatTextView { public MauiTextView(Context context) base(context) { this.ViewAttachedToWindow += MauiTextView_ViewAttachedToWindow; } private void MauiTextView_ViewAttachedToWindow(object? sender, ViewAttachedToWindowEventArgs e) { } //... This was leftover from refactoring, but appeared in dotnet-trace output as 278.55ms (2.4%) mono.android!Android.Views.View.add_ViewAttachedToWindow(System.EventHandler`1<Android.Views.View/ViewAttachedToWindowEv 30.55ms (0.26%) mono.android!Android.Views.View.IOnAttachStateChangeListenerInvoker.n_OnViewAttachedToWindow_Landroid_view_View__mm_wra Where the first is the subscription, and the second is the event firing from Java to C# — only to run an empty managed method. Simply removing this event subscription and empty method, resulted in only a few controls to subscribe to this event as needed 2.76ms (0.02%) mono.android!Android.Views.View.add_ViewAttachedToWindow(System.EventHandler`1<Android.Views.View/ViewAttachedToWindowEv See maui#14833 for details about this improvement. Remove unneeded System.Reflection for {Binding} All bindings in .NET MAUI commonly hit the code path if (property.CanWrite && property.SetMethod.IsPublic && !property.SetMethod.IsStatic) { part.LastSetter = property.SetMethod; var lastSetterParameters = part.LastSetter.GetParameters(); part.SetterType = lastSetterParameters[lastSetterParameters.Length - 1].ParameterType; //... Where ~53% of the time spent applying a binding appeared in dotnet-trace in the MethodInfo.GetParameters() method core.benchmarks!Microsoft.Maui.Benchmarks.BindingBenchmarker.BindName() ... microsoft.maui.controls!Microsoft.Maui.Controls.BindingExpression.SetupPart() System.Private.CoreLib.il!System.Reflection.RuntimeMethodInfo.GetParameters() The above C# is simply finding the property type. It is using a roundabout way of using the property setter’s first parameter, which can be simplified to part.SetterType = property.PropertyType; We could see the results of this change in a BenchmarkDotNet benchmark Method Mean Error StdDev Gen0 Gen1 Allocated –BindName 18.82 us 0.336 us 0.471 us 1.2817 1.2512 10.55 KB ++BindName 18.80 us 0.371 us 0.555 us 1.2512 1.2207 10.23 KB –BindChild 27.47 us 0.542 us 0.827 us 2.0142 1.9836 16.56 KB ++BindChild 26.71 us 0.516 us 0.652 us 1.9226 1.8921 15.94 KB –BindChildIndexer 58.39 us 1.113 us 1.143 us 3.1738 3.1128 26.17 KB ++BindChildIndexer 58.00 us 1.055 us 1.295 us 3.1128 3.0518 25.47 KB Where ++ denotes the new changes. See maui#14830 for further details about this improvement. Use StringComparer.Ordinal for Dictionary and HashSet Profiling a .NET MAUI sample application from a customer, we noticed 4% of the time while scrolling was spent doing dictionary lookups (4.0%) System.Private.CoreLib!System.Collections.Generic.Dictionary<TKey_REF,TValue_REF>.FindValue(TKey_REF) Observing the call stack, some of these were coming from culture-aware string lookups in .NET MAUI microsoft.maui!Microsoft.Maui.PropertyMapper.GetProperty(string) microsoft.maui!Microsoft.Maui.WeakEventManager.AddEventHandler(System.EventHandler<TEventArgs_REF>,string) microsoft.maui!Microsoft.Maui.CommandMapper.GetCommand(string) Which show up in dotnet-trace as a mixture of string comparers (0.98%) System.Private.CoreLib!System.Collections.Generic.NonRandomizedStringEqualityComparer.OrdinalComparer.GetHashCode(string) (0.71%) System.Private.CoreLib!System.String.GetNonRandomizedHashCode() (0.31%) System.Private.CoreLib!System.Collections.Generic.NonRandomizedStringEqualityComparer.OrdinalComparer.Equals(string,stri (0.01%) System.Private.CoreLib!System.Collections.Generic.NonRandomizedStringEqualityComparer.GetStringComparer(object) In cases of Dictionary<string, TValue> or HashSet<string>, we can use StringComparer.Ordinal in many cases to get faster dictionary lookups. This should slightly improve the performance of handlers & all .NET MAUI controls on all platforms. See maui#14900 for details about this improvement. Reduce Java interop in MauiDrawable on Android Profiling a .NET MAUI customer sample while scrolling on a Pixel 5, we saw some interesting time being spent in (0.76%) microsoft.maui!Microsoft.Maui.Graphics.MauiDrawable.OnDraw(Android.Graphics.Drawables.Shapes.Shape,Android.Graphics.Canv (0.54%) microsoft.maui!Microsoft.Maui.Graphics.MauiDrawable.SetDefaultBackgroundColor() This sample has a <Border/> inside a <CollectionView/> and so you can see this work happening while scrolling. Specifically, we reviewed code in .NET MAUI, such as _borderPaint.StrokeWidth = _strokeThickness; _borderPaint.StrokeJoin = _strokeLineJoin; _borderPaint.StrokeCap = _strokeLineCap; _borderPaint.StrokeMiter = _strokeMiterLimit * 2; if (_borderPathEffect != null) _borderPaint.SetPathEffect(_borderPathEffect); This calls from C# to Java five times. Creating a new method in PlatformInterop.java allowed us to reduce it to a single time. We also improved the following method, which would perform many calls from C# to Java // C# void SetDefaultBackgroundColor() { using (var background = new TypedValue()) { if (_context == null || _context.Theme == null || _context.Resources == null) return; if (_context.Theme.ResolveAttribute(globalAndroid.Resource.Attribute.WindowBackground, background, true)) { var resource = _context.Resources.GetResourceTypeName(background.ResourceId); var type = resource?.ToLowerInvariant(); if (type == "color") { var color = new Android.Graphics.Color(ContextCompat.GetColor(_context, background.ResourceId)); _backgroundColor = color; } } } } To be more succinctly implemented in Java as // Java /** * Gets the value of android.R.attr.windowBackground from the given Context * @param context * @return the color or -1 if not found */ public static int getWindowBackgroundColor(Context context) { TypedValue value = new TypedValue(); if (!context.getTheme().resolveAttribute(android.R.attr.windowBackground, value, true) && isColorType(value)) { return value.data; } else { return -1; } } /** * Needed because TypedValue.isColorType() is only API Q+ * https//github.com/aosp-mirror/platform_frameworks_base/blob/1d896eeeb8744a1498128d62c09a3aa0a2a29a16/core/java/android/util/TypedValue.java#L266-L268 * @param value * @return true if the TypedValue is a Color */ private static boolean isColorType(TypedValue value) { if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.Q) { return value.isColorType(); } else { // Implementation from AOSP return (value.type >= TypedValue.TYPE_FIRST_COLOR_INT && value.type <= TypedValue.TYPE_LAST_COLOR_INT); } } Which reduces our new implementation on the C# side to be a single Java call and creation of an Android.Graphics.Color struct void SetDefaultBackgroundColor() { var color = PlatformInterop.GetWindowBackgroundColor(_context); if (color != -1) { _backgroundColor = new Android.Graphics.Color(color); } } After these changes, we instead saw dotnet-trace output, such as (0.28%) microsoft.maui!Microsoft.Maui.Graphics.MauiDrawable.OnDraw(Android.Graphics.Drawables.Shapes.Shape,Android.Graphics.Canv (0.04%) microsoft.maui!Microsoft.Maui.Graphics.MauiDrawable.SetDefaultBackgroundColor() This improves the performance of any <Border/> (and other shapes) on Android, and drops about ~1% of the CPU usage while scrolling in this example. See maui#14933 for further details about this improvement. Improve layout performance of Label on Android Testing various .NET MAUI sample applications on Android, we noticed around 5.1% of time spent in PrepareForTextViewArrange() 1.01s (5.1%) microsoft.maui!Microsoft.Maui.ViewHandlerExtensions.PrepareForTextViewArrange(Microsoft.Maui.IViewHandler,Microsoft.Maui 635.99ms (3.2%) mono.android!Android.Views.View.get_Context() Most of the time is spent just calling Android.Views.View.Context to be able to then call into the extension method internal static int MakeMeasureSpecExact(this Context context, double size) { // Convert to a native size to create the spec for measuring var deviceSize = (int)context!.ToPixels(size); return MeasureSpecMode.Exactly.MakeMeasureSpec(deviceSize); } Calling the Context property can be expensive due the interop from C# to Java. Java returns a handle to the instance, then we have to look up any existing, managed C# objects for the Context. If all this work can simply be avoided, it can improve performance dramatically. In .NET 7, we made overloads to ToPixels() that allows you to get the same value with an Android.Views.View So we can instead do internal static int MakeMeasureSpecExact(this PlatformView view, double size) { // Convert to a native size to create the spec for measuring var deviceSize = (int)view.ToPixels(size); return MeasureSpecMode.Exactly.MakeMeasureSpec(deviceSize); } Not only did this change show improvements in dotnet-trace output, but we saw a noticeable difference in our LOLs per second test application from last year See maui#14980 for details about this improvement. Reduce Java interop calls for controls in .NET MAUI Reviewing the beautiful .NET MAUI “Surfing App” sample by @jsuarezruiz We noticed that a lot of time is spent doing Java interop while scrolling 1.76s (35%) Microsoft.Maui!Microsoft.Maui.Platform.WrapperView.DispatchDraw(Android.Graphics.Canvas) 1.76s (35%) Microsoft.Maui!Microsoft.Maui.Platform.ContentViewGroup.DispatchDraw(Android.Graphics.Canvas) These methods were deeply nested doing interop from Java -> C# -> Java many levels deep. In this case, moving some code from C# to Java could make it where less interop would occur; and in some cases no interop at all! So for example, previously DispatchDraw() was overridden in C# to implement clipping behavior // C# // ContentViewGroup is used internally by many .NET MAUI Controls class ContentViewGroup Android.Views.ViewGroup { protected override void DispatchDraw(Canvas? canvas) { if (Clip != null) ClipChild(canvas); base.DispatchDraw(canvas); } } By creating a PlatformContentViewGroup.java, we can do something like // Java /** * Set by C#, determining if we need to call getClipPath() * @param hasClip */ protected final void setHasClip(boolean hasClip) { this.hasClip = hasClip; postInvalidate(); } @Override protected void dispatchDraw(Canvas canvas) { // Only call into C# if there is a Clip if (hasClip) { Path path = getClipPath(canvas.getWidth(), canvas.getHeight()); if (path != null) { canvas.clipPath(path); } } super.dispatchDraw(canvas); } setHasClip() is called when clipping is enabled/disabled on any .NET MAUI control. This allowed the common path to not interop into C# at all, and only views that have opted into clipping would need to. This is very good because dispatchDraw() is called quite often during Android layout, scrolling, etc. This same treatment was also done to a few other internal .NET MAUI types like WrapperView improving the common case, making interop only occur when views have opted into clipping or drop shadows. For testing the impact of these changes, we used Google’s FrameMetricsAggregator that can be setup in any .NET MAUI application’s Platforms/Android/MainActivity.cs // How often in ms you'd like to print the statistics to the console const int Duration = 1000; FrameMetricsAggregator aggregator; Handler handler; protected override void OnCreate(Bundle savedInstanceState) { base.OnCreate(savedInstanceState); handler = new Handler(Looper.MainLooper); // We were interested in the "Total" time, other metrics also available aggregator = new FrameMetricsAggregator(FrameMetricsAggregator.TotalDuration); aggregator.Add(this); handler.PostDelayed(OnFrame, Duration); } void OnFrame() { // We were interested in the "Total" time, other metrics also available var metrics = aggregator.GetMetrics()[FrameMetricsAggregator.TotalIndex]; int size = metrics.Size(); double sum = 0, count = 0, slow = 0; for (int i = 0; i < size; i++) { int value = metrics.Get(i); if (value != 0) { count += value; sum += i * value; if (i > 16) slow += value; Console.WriteLine($"Frame(s) that took ~{i}ms, count {value}"); } } if (sum > 0) { Console.WriteLine($"Average frame time {sum / count0.00}ms"); Console.WriteLine($"No. of slow frames {slow}"); Console.WriteLine("-----"); } handler.PostDelayed(OnFrame, Duration); } FrameMetricsAggregator‘s API is admittedly a bit odd, but the data we get out is quite useful. The result is basically a lookup table where the key is a duration in milliseconds, and the value is the number of “frames” that took that duration. The idea is any frame that takes longer than 16ms is considered “slow” or “janky” as the Android docs sometimes refer. An example of the .NET MAUI “Surfing App” running on a Pixel 5 Before Frame(s) that took ~4ms, count 1 Frame(s) that took ~5ms, count 6 Frame(s) that took ~6ms, count 10 Frame(s) that took ~7ms, count 12 Frame(s) that took ~8ms, count 10 Frame(s) that took ~9ms, count 6 Frame(s) that took ~10ms, count 1 Frame(s) that took ~11ms, count 2 Frame(s) that took ~12ms, count 4 Frame(s) that took ~13ms, count 2 Frame(s) that took ~15ms, count 1 Frame(s) that took ~16ms, count 1 Frame(s) that took ~18ms, count 2 Frame(s) that took ~19ms, count 1 Frame(s) that took ~20ms, count 5 Frame(s) that took ~21ms, count 2 Frame(s) that took ~22ms, count 1 Frame(s) that took ~25ms, count 1 Frame(s) that took ~32ms, count 1 Frame(s) that took ~34ms, count 1 Frame(s) that took ~60ms, count 1 Frame(s) that took ~62ms, count 1 Frame(s) that took ~63ms, count 1 Frame(s) that took ~64ms, count 2 Frame(s) that took ~66ms, count 1 Frame(s) that took ~67ms, count 1 Frame(s) that took ~68ms, count 1 Frame(s) that took ~69ms, count 2 Frame(s) that took ~70ms, count 2 Frame(s) that took ~71ms, count 2 Frame(s) that took ~72ms, count 1 Frame(s) that took ~73ms, count 2 Frame(s) that took ~74ms, count 2 Frame(s) that took ~75ms, count 1 Frame(s) that took ~76ms, count 1 Frame(s) that took ~77ms, count 2 Frame(s) that took ~78ms, count 3 Frame(s) that took ~79ms, count 1 Frame(s) that took ~80ms, count 1 Frame(s) that took ~81ms, count 1 Average frame time 28.67ms No. of slow frames 43 After the changes to ContentViewGroup and WrapperView were in place, we got a very nice improvement! Even in an app making heavy usage of clipping and shadows After Frame(s) that took ~5ms, count 3 Frame(s) that took ~6ms, count 5 Frame(s) that took ~7ms, count 7 Frame(s) that took ~8ms, count 7 Frame(s) that took ~9ms, count 4 Frame(s) that took ~10ms, count 2 Frame(s) that took ~11ms, count 6 Frame(s) that took ~12ms, count 2 Frame(s) that took ~13ms, count 3 Frame(s) that took ~14ms, count 4 Frame(s) that took ~15ms, count 1 Frame(s) that took ~16ms, count 1 Frame(s) that took ~17ms, count 1 Frame(s) that took ~18ms, count 2 Frame(s) that took ~19ms, count 1 Frame(s) that took ~20ms, count 3 Frame(s) that took ~21ms, count 2 Frame(s) that took ~22ms, count 2 Frame(s) that took ~27ms, count 2 Frame(s) that took ~29ms, count 2 Frame(s) that took ~32ms, count 1 Frame(s) that took ~34ms, count 1 Frame(s) that took ~35ms, count 1 Frame(s) that took ~64ms, count 1 Frame(s) that took ~67ms, count 1 Frame(s) that took ~68ms, count 2 Frame(s) that took ~69ms, count 1 Frame(s) that took ~72ms, count 3 Frame(s) that took ~74ms, count 3 Average frame time 21.99ms No. of slow frames 29 See maui#14275 for further detail about these changes. Improve performance of Entry.MaxLength on Android Investigating a .NET MAUI customer sample Navigating from a Shell flyout. To a new page with several Entry controls. There was a noticeable performance delay. When profiling on a Pixel 5, one “hot path” was Entry.MaxLength 18.52ms (0.22%) microsoft.maui!Microsoft.Maui.Platform.EditTextExtensions.UpdateMaxLength(Android.Widget.EditText,Microsoft.Maui.IEntry) 16.03ms (0.19%) microsoft.maui!Microsoft.Maui.Platform.EditTextExtensions.UpdateMaxLength(Android.Widget.EditText,int) 12.16ms (0.14%) microsoft.maui!Microsoft.Maui.Platform.EditTextExtensions.SetLengthFilter(Android.Widget.EditText,int) EditTextExtensions.UpdateMaxLength() calls EditText.Text getter and setter EditTextExtensions.SetLengthFilter() calls EditText.Get/SetFilters() What happens is we end up marshaling strings and IInputFilter[] back and forth between C# and Java for every Entry control. All Entry controls go through this code path (even ones with a default value for MaxLength), so it made sense to move some of this code from C# to Java instead. Our C# code before // C# public static void UpdateMaxLength(this EditText editText, int maxLength) { editText.SetLengthFilter(maxLength); var newText = editText.Text.TrimToMaxLength(maxLength); if (editText.Text != newText) editText.Text = newText; } public static void SetLengthFilter(this EditText editText, int maxLength) { if (maxLength == -1) maxLength = int.MaxValue; var currentFilters = new List<IInputFilter>(editText.GetFilters() ?? new IInputFilter[0]); var changed = false; for (var i = 0; i < currentFilters.Count; i++) { if (currentFilters[i] is InputFilterLengthFilter) { currentFilters.RemoveAt(i); changed = true; break; } } if (maxLength >= 0) { currentFilters.Add(new InputFilterLengthFilter(maxLength)); changed = true; } if (changed) editText.SetFilters(currentFilters.ToArray()); } Moved to Java (with identical behavior) instead // Java /** * Sets the maxLength of an EditText * @param editText * @param maxLength */ public static void updateMaxLength(@NonNull EditText editText, int maxLength) { setLengthFilter(editText, maxLength); if (maxLength < 0) return; Editable currentText = editText.getText(); if (currentText.length() > maxLength) { editText.setText(currentText.subSequence(0, maxLength)); } } /** * Updates the InputFilter[] of an EditText. Used for Entry and SearchBar. * @param editText * @param maxLength */ public static void setLengthFilter(@NonNull EditText editText, int maxLength) { if (maxLength == -1) maxLength = Integer.MAX_VALUE; List<InputFilter> currentFilters = new ArrayList<>(Arrays.asList(editText.getFilters())); boolean changed = false; for (int i = 0; i < currentFilters.size(); i++) { InputFilter filter = currentFilters.get(i); if (filter instanceof InputFilter.LengthFilter) { currentFilters.remove(i); changed = true; break; } } if (maxLength >= 0) { currentFilters.add(new InputFilter.LengthFilter(maxLength)); changed = true; } if (changed) { InputFilter[] newFilter = new InputFilter[currentFilters.size()]; editText.setFilters(currentFilters.toArray(newFilter)); } } This avoids marshaling (copying!) string and array values back and forth from C# to Java. With these changes in place, the calls to EditTextExtensions.UpdateMaxLength() are now so fast they are missing completely from dotnet-trace output, saving ~19ms when navigating to the page in the customer sample. See maui#15614 for details about this improvement. Improve memory usage of CollectionView on Windows We reviewed a .NET MAUI customer sample with a CollectionView of 150,000 data-bound rows. Debugging what happens at runtime, .NET MAUI was effectively doing _itemTemplateContexts = new List<ItemTemplateContext>(capacity 150_000); for (int n = 0; n < 150_000; n++) { _itemTemplateContexts.Add(null); } And then each item is created as it is scrolled into view if (_itemTemplateContexts[index] == null) { _itemTemplateContexts[index] = context = new ItemTemplateContext(...); } return _itemTemplateContexts[index]; This wasn’t the best approach, but to improve things use a Dictionary<int, T> instead, just let it size dynamically. use TryGetValue(..., out var context), so each call accesses the indexer one less time than before. use either the bound collection’s size or 64 (whichever is smaller) as a rough estimate of how many might fit on screen at a time Our code changes to if (!_itemTemplateContexts.TryGetValue(index, out var context)) { _itemTemplateContexts[index] = context = new ItemTemplateContext(...); } return context; With these changes in place, a memory snapshot of the app after startup Before Heap Size 82,899.54 KB After Heap Size 81,768.76 KB Which is saving about 1MB of memory on launch. In this case, it feels better to just let the Dictionary size itself with an estimate of what capacity will be. See maui#16838 for details about this improvement. Use UnmanagedCallersOnlyAttribute on Apple platforms When unmanaged code calls into managed code, such as invoking a callback from Objective-C, the [MonoPInvokeCallbackAttribute] was previously used in Xamarin.iOS, Xamarin.Mac, and .NET 6+ for this purpose. The [UnmanagedCallersOnlyAttribute] attribute came along as a modern replacement for this Mono feature, which is implemented in a way with performance in mind. Unfortunately, there are a few restrictions when using this new attribute Method must be marked static. Must not be called from managed code. Must only have blittable arguments. Must not have generic type parameters or be contained within a generic class. Not only did we have to refactor the “code generator” that produces many of the bindings for Apple APIs for AppKit, UIKit, etc., but we also had many manual bindings that would need the same treatment. The end result is that most callbacks from Objective-C to C# should be faster in .NET 8 than before. See xamarin-macios#10470 and xamarin-macios#15783 for details about these improvements. Faster Java interop for strings on Android When binding members which have parameter types or return types which are java.lang.CharSequence, the member is “overloaded” to replace CharSequence with System.String, and the “original” member has a Formatted suffix. For example, consider android.widget.TextView, which has getText() and setText() methods which have parameter types and return types which are java.lang.CharSequence // Java class TextView extends View { public CharSequence getText(); public final void setText(CharSequence text); } When bound, this results in two properties // C# class TextView View { public Java.Lang.ICharSequence? TextFormatted { get; set; } public string? Text { get; set; } } The “non-Formatted overload” works by creating a temporary String object to invoke the Formatted overload, so the actual implementation looks like partial class TextView { public string? Text { get => TextFormatted?.ToString (); set { var jls = value == null ? null new Java.Lang.String (value); TextFormatted = jls; jls?.Dispose (); } } } TextView.Text is much easer to understand & simpler to consume for .NET developers than TextView.TextFormatted. A problem with the this approach is performance creating a new Java.Lang.String instance requires Creating the managed peer (the Java.Lang.String instance), Creating the native peer (the java.lang.String instance), And registering the mapping between (1) and (2) And then immediately use and dispose the value… This is particularly noticeable with .NET MAUI apps. Consider a customer sample, which uses XAML to set data-bound Text values in a CollectionView, which eventually hit TextView.Text. Profiling shows 653.69ms (6.3%) mono.android!Android.Widget.TextView.set_Text(string) 198.05ms (1.9%) mono.android!Java.Lang.String..ctor(string) 121.57ms (1.2%) mono.android!Java.Lang.Object.Dispose() 6.3% of scrolling time is spent in the TextView.Text property setter! Partially optimize this case if the *Formatted member is (1) a property, and (2) not virtual, then we can directly call the Java setter method. This avoids the need to create a managed peer and to register a mapping between the peers partial class TextView { public string? Text { get => TextFormatted?.ToString (); // unchanged set { const string __id = "setText.(Ljava/lang/CharSequence;)V"; JniObjectReference native_value = JniEnvironment.Strings.NewString (value); try { JniArgumentValue* __args = stackalloc JniArgumentValue [1]; __args [0] = new JniArgumentValue (native_value); _members.InstanceMethods.InvokeNonvirtualVoidMethod (__id, this, __args); } finally { JniObjectReference.Dispose (ref native_value); } } } } With the result being Method Mean Error StdDev Allocated Before SetFinalText 6.632 us 0.0101 us 0.0079 us 112 B After SetFinalText 1.361 us 0.0022 us 0.0019 us – The TextView.Text property setter invocation time is reduced to 20% of the previous average invocation time. Note that the virtual case is problematic for other reasons, but luckily enough TextView.setText() is non-virtual and likely one of the more commonly used Android APIs. See java-interop#1101 for details about this improvement. Faster Java interop for C# events on Android Profiling a .NET MAUI customer sample while scrolling on a Pixel 5, We saw ~2.2% of the time spent in the IOnFocusChangeListenerImplementor constructor, due to a subscription to the View.FocusChange event (2.2%) mono.android!Android.Views.View.IOnFocusChangeListenerImplementor..ctor() MAUI subscribes to Android.Views.View.FocusChange for every view placed on the screen, which happens while scrolling in this sample. Reviewing the generated code for the IOnFocusChangeListenerImplementor constructor, we see it still uses outdated JNIEnv APIs public IOnFocusChangeListenerImplementor () base ( Android.Runtime.JNIEnv.StartCreateInstance ("mono/android/view/View_OnFocusChangeListenerImplementor", "()V"), JniHandleOwnership.TransferLocalRef ) { Android.Runtime.JNIEnv.FinishCreateInstance (((Java.Lang.Object) this).Handle, "()V"); } Which we can change to use the newer/faster Java.Interop APIs public unsafe IOnFocusChangeListenerImplementor () base (IntPtr.Zero, JniHandleOwnership.DoNotTransfer) { const string __id = "()V"; if (((Java.Lang.Object) this).Handle != IntPtr.Zero) return; var h = JniPeerMembers.InstanceMethods.StartCreateInstance (__id, ((object) this).GetType (), null); SetHandle (h.Handle, JniHandleOwnership.TransferLocalRef); JniPeerMembers.InstanceMethods.FinishCreateInstance (__id, this, null); } These are better because the equivalent call to JNIEnv.FindClass() is cached, among other things. This was just one of the cases that was accidentally missed when we implemented the new Java.Interop APIs in the Xamarin timeframe. We simply needed to update our code generator to emit a better C# binding for this case. After these changes, we saw instead results in dotnet-trace (0.81%) mono.android!Android.Views.View.IOnFocusChangeListenerImplementor..ctor() This should improve the performance of all C# events that wrap Java listeners, a design-pattern commonly used in Java and Android applications. This includes the FocusedChanged event used by all .NET MAUI views on Android. See java-interop#1105 for details about this improvement. Use Function Pointers for JNI There is various machinery and generated code that makes Java interop possible from C#. Take, for example, the following instance method foo() in Java // Java object foo(object bar) { // returns some value } A C# method named CallObjectMethod is responsible for calling Java’s Native Interface (JNI) that calls into the JVM to actually invoke the Java method public static unsafe JniObjectReference CallObjectMethod (JniObjectReference instance, JniMethodInfo method, JniArgumentValue* args) { //... IntPtr thrown; var tmp = NativeMethods.java_interop_jnienv_call_object_method_a (JniEnvironment.EnvironmentPointer, out thrown, instance.Handle, method.ID, (IntPtr) args); Exception __e = JniEnvironment.GetExceptionForLastThrowable (thrown); if (__e != null) ExceptionDispatchInfo.Capture (__e).Throw (); JniEnvironment.LogCreateLocalRef (tmp); return new JniObjectReference (tmp, JniObjectReferenceType.Local); } In Xamarin.Android, .NET 6, and .NET 7 all calls into Java went through a java_interop_jnienv_call_object_method_a p/invoke, which signature looks like [DllImport (JavaInteropLib, CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)] internal static extern unsafe jobject java_interop_jnienv_call_object_method_a (IntPtr jnienv, out IntPtr thrown, jobject instance, IntPtr method, IntPtr args); Which is implemented in C as JI_API jobject java_interop_jnienv_call_object_method_a (JNIEnv *env, jthrowable *_thrown, jobject instance, jmethodID method, jvalue* args) { *_thrown = 0; jobject _r_ = (*env)->CallObjectMethodA (env, instance, method, args); *_thrown = (*env)->ExceptionOccurred (env); return _r_; } C# 9 introduced function pointers that allowed us a way to simplify things slightly — and make them faster as a result. So instead of using p/invoke in .NET 8, we could instead call a new unsafe method named CallObjectMethodA // Before var tmp = NativeMethods.java_interop_jnienv_call_object_method_a (JniEnvironment.EnvironmentPointer, out thrown, instance.Handle, method.ID, (IntPtr) args); // After var tmp = JniNativeMethods.CallObjectMethodA (JniEnvironment.EnvironmentPointer, instance.Handle, method.ID, (IntPtr) args); Which calls a C# function pointer directly [System.Runtime.CompilerServices.MethodImpl (System.Runtime.CompilerServices.MethodImplOptions.AggressiveInlining)] internal static unsafe jobject CallObjectMethodA (IntPtr env, jobject instance, IntPtr method, IntPtr args) { return (*((JNIEnv**)env))->CallObjectMethodA (env, instance, method, args); } This function pointer declared using the new syntax introduced in C# 9 public delegate* unmanaged <IntPtr, jobject, IntPtr, IntPtr, jobject> CallObjectMethodA; Comparing the two implementations with a manual benchmark # JIPinvokeTiming timing 000001.6993644 # Average Invocation 0.00016993643999999998ms # JIFunctionPointersTiming timing 000001.6561349 # Average Invocation 0.00016561349ms With a Release build, the average invocation time for JIFunctionPointersTiming takes 97% of the time as JIPinvokeTiming, i.e. is 3% faster. Additionally, using C# 9 function pointers means we can get rid of all of the java_interop_jnienv_*() C functions, which shrinks libmonodroid.so by ~55KB for each architecture. See xamarin-android#8234 and java-interop#938 for details about this improvement. Removed Xamarin.AndroidX.Legacy.Support.V4 Reviewing .NET MAUI’s Android dependencies, we noticed a suspicious package Xamarin.AndroidX.Legacy.Support.V4 If you are familiar with the Android Support Libraries, these are a set of packages Google provides to “polyfill” APIs to past versions of Android. This gives them a way to bring new APIs to old OS versions, since the Android ecosystem (OEMs, etc.) are much slower to upgrade as compared to iOS, for example. This particular package, Legacy.Support.V4, is actually support for Android as far back as Android API 4! The minimum supported Android version in .NET is Android API 21, which was released in 2017. It turns out this dependency was brought over from Xamarin.Forms and was not actually needed. As expected from this change, lots of Java code was removed from .NET MAUI apps. So much, in fact, that .NET 8 MAUI applications are now under the multi-dex limit — all Dalvik bytecode can fix into a single classes.dex file. A detailed breakdown of the size changes using apkdiff > apkdiff -f com.companyname.maui_before-Signed.apk com.companyname.maui_after-Signed.apk Size difference in bytes ([*1] apk1 only, [*2] apk2 only) + 1,598,040 classes.dex - 6 META-INF/androidx.asynclayoutinflater_asynclayoutinflater.version *1 - 6 META-INF/androidx.legacy_legacy-support-core-ui.version *1 - 6 META-INF/androidx.legacy_legacy-support-v4.version *1 - 6 META-INF/androidx.media_media.version *1 - 455 assemblies/assemblies.blob - 564 res/layout/notification_media_action.xml *1 - 744 res/layout/notification_media_cancel_action.xml *1 - 1,292 res/layout/notification_template_media.xml *1 - 1,584 META-INF/BNDLTOOL.SF - 1,584 META-INF/MANIFEST.MF - 1,696 res/layout/notification_template_big_media.xml *1 - 1,824 res/layout/notification_template_big_media_narrow.xml *1 - 2,456 resources.arsc - 2,756 res/layout/notification_template_media_custom.xml *1 - 2,872 res/layout/notification_template_lines_media.xml *1 - 3,044 res/layout/notification_template_big_media_custom.xml *1 - 3,216 res/layout/notification_template_big_media_narrow_custom.xml *1 - 2,030,636 classes2.dex Summary - 24,111 Other entries -0.35% (of 6,880,759) - 432,596 Dalvik executables -3.46% (of 12,515,440) + 0 Shared libraries 0.00% (of 12,235,904) - 169,179 Package size difference -1.12% (of 15,123,185) See dotnet/maui#12232 for details about this improvement. Deduplication of generics on iOS and macOS In .NET 7, iOS applications experienced app size increases due to C# generics usage across multiple .NET assemblies. When the .NET 7 Mono AOT compiler encounters a generic instance that is not handled by generic sharing, it will emit code for the instance. If the same instance is encountered during AOT compilation in multiple assemblies, the code will be emitted multiple times, increasing code size. In .NET 8, new dedup-skip and dedup-include command-line options are passed to the Mono AOT compiler. A new aot-instances.dll assembly is created for sharing this information in one place throughout the application. The change was tested on MySingleView app and Monotouch tests in the xamarin/xamarin-macios codebase App Baseline size on disk .ipa (MB) Target size on disk .ipa (MB) Baseline size on disk .app (MB) Target size on disk .app (MB) Baseline build time (s) Target build time (s) .app diff (%) MySingleView Release iOS 5.4 5.4 29.2 15.2 29.2 16.8 47.9 MySingleView Release iOSSimulator-arm64 N/A N/A 469.5 341.8 468.0 330.0 27.2 Monotouch Release llvm iOS 49.0 38.8 209.6 157.4 115.0 130.0 24.9 See xamarin-macios#17766 for details about this improvement. Fix System.Linq.Expressions implementation on iOS-like platforms In .NET 7, codepaths in System.Linq.Expressions were controlled by various flags such as CanCompileToIL CanEmitObjectArrayDelegate CanCreateArbitraryDelegates These flags were controlling codepaths which are “AOT friendly” and those that are not. For desktop platforms, NativeAOT specifies the following configuration for AOT-compatible code <IlcArg Include="--featureSystem.Linq.Expressions.CanCompileToIL=false" /> <IlcArg Include="--featureSystem.Linq.Expressions.CanEmitObjectArrayDelegate=false" /> <IlcArg Include="--featureSystem.Linq.Expressions.CanCreateArbitraryDelegates=false" /> When it comes to iOS-like platforms, System.Linq.Expressions library was built with constant propagation enabled and control variables were removed. This further caused above-listed NativeAOT feature switches not to have any effect (fail to trim during app build), potentially causing the AOT compilation to follow unsupported code paths on these platforms. In .NET8, we have unified the build of System.Linq.Expressions.dll shipping the same assembly for all supported platforms and runtimes, and simplified these switches to respect IsDynamicCodeSupported so that the .NET trimmer can remove the appropriate IL in System.Linq.Expressions.dll at application build time. See dotnet/runtime#87924 and dotnet/runtime#89308 for details about this improvement. Set DynamicCodeSupport=false for iOS and Catalyst In .NET 8, the feature switch $(DynamicCodeSupport) is set to false for platforms Where it is not possible to publish without the AOT compiler. When interpreter is not enabled. Which boils down to applications running on iOS, tvOS, MacCatalyst, etc. DynamicCodeSupport=false enables the .NET trimmer to remove code paths depending on RuntimeFeature.IsDynamicCodeSupported such as this example in System.Linq.Expressions. Estimated size savings are dotnet new maui (ios) old SLE.dll new SLE.dll + DynamicCodeSupported=false diff (%) Size on disk (Mb) 40,53 38,78 -4,31% .pkg (Mb) 14,83 14,20 -4,21% When combined with the System.Linq.Expressions improvements on iOS-like platforms, this showed a nice overall improvement to application size See xamarin-macios#18555 for details about this improvement. Memory Leaks Memory Leaks and Quality Given that the major theme for .NET MAUI in .NET 8 is quality, memory-related issues became a focal point for this release. Some of the problems found existed even in the Xamarin.Forms codebase, so we are happy to work towards a framework that developers can rely on for their cross-platform .NET applications. For full details on the work completed in .NET 8, we’ve various PRs and Issues related to memory issues at Pull Requests Issues You can see that considerable progress was made in .NET 8 in this area. If we compare .NET 7 MAUI versus .NET 8 MAUI in a sample application running on Windows, displaying the results of GC.GetTotalMemory() on screen Then compare the sample application running on macOS, but with many more pages pushed onto the navigation stack See the sample code for this project on GitHub for further details. Diagnosing leaks in .NET MAUI The symptom of a memory leak in a .NET MAUI application, could be something like Navigate from the landing page to a sub page. Go back. Navigate to the sub page again. Repeat. Memory grows consistently until the OS closes the application due to lack of memory. In the case of Android, you may see log messages such as 07-07 185139.090 17079 17079 D Mono GC_MAJOR (user request) time 137.21ms, stw 140.60ms los size 10984K in use 3434K 07-07 185139.090 17079 17079 D Mono GC_MAJOR_SWEEP major size 116192K in use 108493K 07-07 185139.092 17079 17079 I monodroid-gc 46204 outstanding GREFs. Performing a full GC! In this example, a 116MB heap is quite large for a mobile application, as well as over 46,000 C# <-> Java wrapper objects! To truly determine if the sub page is leaking, we can make a couple modifications to a .NET MAUI application Add logging in a finalizer. For example ~MyPage() => Console.WriteLine("Finalizer for ~MyPage()"); While navigating through your app, you can find out if entire pages are living forever if the log message is never displayed. This is a common symptom of a leak, because any View holds .Parent.Parent.Parent, etc. all the way up to the Page object. Call GC.Collect() somewhere in the app, such as the sub page’s constructor public MyPage() { GC.Collect(); // For debugging purposes only, remove later InitializeComponent(); } This makes the GC more deterministic, in that we are forcing it to run more frequently. Each time we navigate to the sub page, we are more likely causing the old sub page’s to go away. If things are working properly, we should see the log message from the finalizer. Note GC.Collect() is for debugging purposes only. You should not need this in your app after investigation is complete, so be sure to remove it afterward. With these changes in place, test a Release build of your app. On iOS, Android, macOS, etc. you can watch console output of your app to determine what is actually happening at runtime. adb logcat, for example, is a way to view these logs on Android. If running on Windows, you can also use Debug > Windows > Diagnostic Tools inside Visual Studio to take memory snapshots inside Visual Studio. In the future, we would like Visual Studio’s diagnostic tooling to support .NET MAUI applications running on other platforms. See our memory leaks wiki page for more information related to memory leaks in .NET MAUI applications. Patterns that cause leaks C# events C# events, just like a field, property, etc. can create strong references between objects. Let’s look at a situation where things can go wrong. Take for example, the cross-platform Grid.ColumnDefinitions property public class Grid Layout, IGridLayout { public static readonly BindableProperty ColumnDefinitionsProperty = BindableProperty.Create("ColumnDefinitions", typeof(ColumnDefinitionCollection), typeof(Grid), null, validateValue (bindable, value) => value != null, propertyChanged UpdateSizeChangedHandlers, defaultValueCreator bindable => { var colDef = new ColumnDefinitionCollection(); colDef.ItemSizeChanged += ((Grid)bindable).DefinitionsChanged; return colDef; }); public ColumnDefinitionCollection ColumnDefinitions { get { return (ColumnDefinitionCollection)GetValue(ColumnDefinitionsProperty); } set { SetValue(ColumnDefinitionsProperty, value); } } Grid has a strong reference to its ColumnDefinitionCollection via the BindableProperty. ColumnDefinitionCollection has a strong reference to Grid via the ItemSizeChanged event. If you put a breakpoint on the line with ItemSizeChanged +=, you can see the event has an EventHandler object where the Target is a strong reference back to the Grid. In some cases, circular references like this are completely OK. The .NET runtime(s)’ garbage collectors know how to collect cycles of objects that point each other. When there is no “root” object holding them both, they can both go away. The problem comes in with object lifetimes what happens if the ColumnDefinitionCollection lives for the life of the entire application? Consider the following Style in Application.Resources or Resources/Styles/Styles.xaml <Style TargetType="Grid" xKey="GridStyleWithColumnDefinitions"> <Setter Property="ColumnDefinitions" Value="18,*"/> </Style> If you applied this Style to a Grid on a random Page Application‘s main ResourceDictionary holds the Style. The Style holds a ColumnDefinitionCollection. The ColumnDefinitionCollection holds the Grid. Grid unfortunately holds the Page via .Parent.Parent.Parent, etc. This situation could cause entire Page‘s to live forever! Note The issue with Grid is fixed in maui#16145, but is an excellent example of illustrating how C# events can go wrong. Circular references on Apple platforms Even since the early days of Xamarin.iOS, there has existed an issue with “circular references” even in a garbage-collected runtime like .NET. C# objects co-exist with a reference-counted world on Apple platforms, and so a C# object that subclasses NSObject can run into situations where they can accidentally live forever — a memory leak. This is not a .NET-specific problem, as you can just as easily create the same situation in Objective-C or Swift. Note that this does not occur on Android or Windows platforms. Take for example, the following circular reference class MyViewSubclass UIView { public UIView? Parent { get; set; } public void Add(MyViewSubclass subview) { subview.Parent = this; AddSubview(subview); } } //... var parent = new MyViewSubclass(); var view = new MyViewSubclass(); parent.Add(view); In this case parent -> view via Subviews view -> parent via the Parent property The reference count of both objects is non-zero. Both objects live forever. This problem isn’t limited to a field or property, you can create similar situations with C# events class MyView UIView { public MyView() { var picker = new UIDatePicker(); AddSubview(picker); picker.ValueChanged += OnValueChanged; } void OnValueChanged(object? sender, EventArgs e) { } // Use this instead and it doesn't leak! //static void OnValueChanged(object? sender, EventArgs e) { } } In this case MyView -> UIDatePicker via Subviews UIDatePicker -> MyView via ValueChanged and EventHandler.Target Both objects live forever. A solution for this example, is to make OnValueChanged method static, which would result in a null Target on the EventHandler instance. Another solution, would be to put OnValueChanged in a non-NSObject subclass class MyView UIView { readonly Proxy _proxy = new(); public MyView() { var picker = new UIDatePicker(); AddSubview(picker); picker.ValueChanged += _proxy.OnValueChanged; } class Proxy { public void OnValueChanged(object? sender, EventArgs e) { } } } This is the pattern we’ve used in most .NET MAUI handlers and other UIView subclasses. See the MemoryLeaksOniOS sample repo, if you would like to play with some of these scenarios in isolation in an iOS application without .NET MAUI. Roslyn analyzer for Apple platforms We also have an experimental Roslyn Analyzer that can detect these situations at build time. To add it to net7.0-ios, net8.0-ios, etc. projects, you can simply install a NuGet package <PackageReference Include="MemoryAnalyzers" Version="0.1.0-beta.3" PrivateAssets="all" /> Some examples of a warning would be public class MyView UIView { public event EventHandler MyEvent; } Event 'MyEvent' could could memory leaks in an NSObject subclass. Remove the event or add the [UnconditionalSuppressMessage("Memory", "MA0001")] attribute with a justification as to why the event will not leak. Note that the analyzer can warns if there might be an issue, so it can be quite noisy to enable in a large, existing codebase. Inspecting memory at runtime is the best way to determine if there is truly a memory leak. Tooling and Documentation Simplified dotnet-trace and dotnet-dsrouter In .NET 7, profiling a mobile application was a bit of a challenge. You had to run dotnet-dsrouter and dotnet-trace together and get all the settings right to be able to retrieve a .nettrace or speedscope file for performance investigations. There was also no built-in support for dotnet-gcdump to connect to dotnet-dsrouter to get memory snapshots of a running .NET MAUI application. In .NET 8, we’ve streamlined this scenario by making new commands for dotnet-dsrouter that simplifies the workflow. To verify you have the latest diagnostic tooling, you can install them via $ dotnet tool install -g dotnet-dsrouter You can invoke the tool using the following command dotnet-dsrouter Tool 'dotnet-dsrouter' was successfully installed. $ dotnet tool install -g dotnet-gcdump You can invoke the tool using the following command dotnet-gcdump Tool 'dotnet-gcdump' was successfully installed. $ dotnet tool install -g dotnet-trace You can invoke the tool using the following command dotnet-trace Tool 'dotnet-trace' was successfully installed. Verify you have at least 8.x versions of these tools $ dotnet tool list -g Package Id Version Commands -------------------------------------------------------------------------------------- dotnet-dsrouter 8.0.452401 dotnet-dsrouter dotnet-gcdump 8.0.452401 dotnet-gcdump dotnet-trace 8.0.452401 dotnet-trace To profile an Android application on an Android emulator, first build and install your application in Release mode such as $ dotnet build -f net8.0-android -tInstall -c Release -pAndroidEnableProfiler=true Build SUCCEEDED. 0 Warning(s) 0 Error(s) Next, open a terminal to run dotnet-dsrouter $ dotnet-dsrouter android-emu Start an application on android emulator with one of the following environment variables set DOTNET_DiagnosticPorts=10.0.2.29000,nosuspend,connect DOTNET_DiagnosticPorts=10.0.2.29000,suspend,connect Then in a second terminal window, we can set the debug.mono.profile Android system property, as the stand-in for $DOTNET_DiagnosticPorts $ adb shell setprop debug.mono.profile '10.0.2.29000,suspend,connect' $ dotnet-trace ps 3248 dotnet-dsrouter $ dotnet-trace collect -p 3248 --format speedscope ... [00000009] Recording trace 3.2522 (MB) Press <Enter> or <Ctrl+C> to exit... Note Android doesn’t have good support for environment variables like $DOTNET_DiagnosticPorts. You can create an AndroidEnvironment text file for setting environment variables, but Android system properties can be simpler as they would not require rebuilding the application to set them. Upon launching the Android application, it should be able to connect to dotnet-dsrouter -> dotnet-trace and record performance profiling information for investigation. The --format argument is optional and it defaults to .nettrace. However, .nettrace files can be viewed only with Perfview on Windows, while the speedscope JSON files can be viewed “on” macOS or Linux by uploading them to https//speedscope.app. Note When providing a process ID to dotnet-trace, it knows how to tell if a process ID is dotnet-dsrouter and connect through it appropriately. dotnet-dsrouter has the following new commands to simplify the workflow dotnet-dsrouter android Android devices dotnet-dsrouter android-emu Android emulators dotnet-dsrouter ios iOS devices dotnet-dsrouter ios-sim iOS simulators See the .NET MAUI wiki for more information about profiling .NET MAUI applications on each platform. dotnet-gcdump Support for Mobile In .NET 7, we had a somewhat complex method (see wiki) for getting a memory snapshot of an application on the Mono runtime (such as iOS or Android). You had to use a Mono-specific event provider such as dotnet-trace collect --diagnostic-port /tmp/maui-app --providers Microsoft-DotNETRuntimeMonoProfiler0xC9000014 And then we relied on Filip Navara’s mono-gcdump tool (thanks Filip!) to convert the .nettrace file to .gcdump to be opened in Visual Studio or PerfView. In .NET 8, we now have dotnet-gcdump support for mobile scenarios. If you want to get a memory snapshot of a running application, you can use dotnet-gcdump in a similar fashion as dotnet-trace $ dotnet-gcdump ps 3248 dotnet-dsrouter $ dotnet-gcdump collect -p 3248 Writing gcdump to '20231018_115631_29880.gcdump'... Note This requires the exact same setup as dotnet-trace, such as -pAndroidEnableProfiler=true, dotnet-dsrouter, adb commands, etc. This greatly streamlines our workflow for investigating memory leaks in .NET MAUI applications. See our memory leaks wiki page for more information. The post .NET 8 Performance Improvements in .NET MAUI appeared first on .NET Blog.

Numerical Linear Algebra for Programmers | New Boo ...

Asp.Net Core Error: InvalidOperationException: The ...

ASP.NET 8 Best Practices: Coding, Performance Tips ...

Windows 10 Linux Subsystem- Develop with Ruby on R ...

Help The Hungry Video chat with Jose Mourinho and ...

Performance Tuning for ASP.NET Web Applications

How to Play Pokemon TCG Past, Present and Future - ...

Migrate Existing Asp.Net Web Site to .Net Core

Mit YouTube-Tutorial Indische Studenten operieren ...

Profile Common Error Asp.Net

MSBUILD : error MSB1009: Project file does not exi ...

Linux has been ported to run on Apple’s M1 Macs

How to Enable COM Ports on Windows 10 or 11 and Us ...

How to get a path to a file in an ASP Dot Net (ASP ...

Destiny 2 getting new tutorial experience, teasing ...

Return url is required

A Data CEO’s Guide to Becoming a Data Scientist Fr ...

ASP.Net Scheduled Jobs (Fire and Forget)

Bella Hadid's no-tie bun tutorial leaves the inter ...

XmlException: The ':' character, hexadecimal value ...

r/memes - You know it’s a good tutorial when the v ...

How to get a base url for Asp.Net Core Application ...

Asp.Net Core 3.1 Not Returning NoContent() or BadR ...

Publishing an F-Droid Repo Using a Hardware Securi ...

Learn new technology through writing a tutorial ab ...

How to Write to PDF using an Open Source Library c ...

Youtube's ban on "hacking techniques" threatens to ...

From Zero to main() How to Write a Bootloader fro ...

error : MSB4803: The task "ResolveComReference" is ...

Your First LTE

Computer Vision in PyTorch (Part 2) Preparing Dat ...

Gigi Hadid Discusses Plastic Surgery And Pregnancy ...

Put Database Offline sql Server (Move DB)

User Control (ASP.Net)

A love letter to Eve Online's tutorial

Learn Python in 2018

Paige Spiranac gets into the holiday spirit in San ...

InkScape Tutorial

Kick Serve Transformation in 10 Minutes - Tennis S ...

Paige Spiranac gets into the holiday spirit in San ...

Introducing Cheekmate, the wireless haptic communi ...

The Settlers’ 2022 release date and closed beta an ...

How to get a User ID in Asp.Net Core 3.1 (ClaimPri ...

How do I design a game from scratch? | Team Avocad ...

Asp.Net Core 3.1 Error: FromSqlRaw or FromSqlInter ...

Bella Hadid's no-tie bun tutorial leaves the inter ...

ASP.NET Core 3.1 Get the Path to the Images Folder ...

ASP.Net Core 2.2 Razor View Routing for Partial Vi ...

Help the Hungry Win video chat with Jose Mourinho ...

Profile Common Error Asp.Net

Application startup exception: System.InvalidOpera ...

Why Breath of the Wild's Great Plateau Is Gaming's ...

AWS Inferentia2 builds on AWS Inferentia1 by deliv ...

Sullivan County business and municipal leaders giv ...

ASP.Net Core 2.2 MVC Error g.cshtml.cs: The type o ...

90 Day Fiance alum Miona Bell shows off beauty ski ...

Spacebase Startopia - Gameplay Trailer - IGN

Asp.Net Core MVC Razor View Form not posting data ...

Debugging ASP.NET Core Applications: Accessing Kes ...

Apple brings its free tutorials to YouTube | Engad ...

6 Reasons Breath of the Wild is Still Riveting in ...

Asp.Net MVC Development Notes

Getting started with C programming a lightning-fa ...

Here’s a step-by-step tutorial for getting your fi ...

[Solved]: Asp.Net Core Error 'byte[]' does not co ...

Asp.Net Core Error: InvalidOperationException: The ...

File Archiving and Compression Commands on Linux

Understanding ASP.Net Core 2.2 Database Context C ...

Adventuron – Text Adventure Authoring in the Brows ...

Artist Sees Into The System

DNS Host File Not Working (ASP.Net Core 3.1)

Microsoft.Common.CurrentVersion.targets(4678,5): e ...

This $20 FL Studio training is the perfect introdu ...

What’s new with identity in .NET 8

Create and Verify PASETO Tokens in Java

ASP.NET Tracing

.NET 8 Performance Improvements in .NET MAUI

Asp.Net Core 3.1 Error The 'inject' directive exp ...

Tyson Fury posts tutorial video for Anthony Joshua ...

Computer Vision in PyTorch (Part 1) Building Your ...