Staff

Abstract:
Visual inspection of civil infrastructure for structural health assessment, as performed by structural engineers, is expensive and time-consuming. Therefore, automating this process is highly attractive, which has received significant attention in recent years. With the increasing capabilities of computers, deep neural networks have become a standard tool and can be used for structural health inspections. A key challenge, however, is the availability of reliable datasets. In this work, the U-net and DeepLab v3+ convolutional neural networks are trained on a synthetic Tokaido dataset. This dataset comprises images representative of data acquired by unmanned aerial vehicle (UAV) imagery and corresponding ground truth data. The data includes semantic segmentation masks for both categorizing structural elements (slabs, beams, and columns) and assessing structural damage (concrete spalling or exposed rebars). Data augmentation, including both image quality degradation (e.g., brightness modification, added noise) and image transformations (e.g., image flipping), is applied to the synthetic dataset. The selected neural network architectures achieve excellent performance, reaching values of 97% for accuracy and 87% for Mean Intersection over Union (mIoU) on the validation data. It also demonstrates promising results in the semantic segmentation of real-world structures captured in photographs, despite being trained solely on synthetic data. Additionally, based on the obtained results of semantic segmentation, it can be concluded that DeepLabV3+ outperforms U-net in structural component identification. However, this is not the case in the damage identification task.

Keywords:
semantic segmentation, structural health monitoring, computer vision-based techniques

Abstract:
Phased-Array Ultrasonic Technique is traditionally used for the non-destructive inspection of welds and supported by industrial-grade inspection equipment. FullMatrix Capture (FMC) with Total Focusing Method (TFM) provide new capabilities and multimodal imaging, but available commercial scanners have limitations in acquisition speed (30–300MB/s) and reconstruction speed. Our goal was to develop a solution for FMC acquisition that can be applied to high-speed robotized weld scanning (speed of 100 mm/s with a resolution of 1 mm). For FMC acquisition, we have applied a portable programmable ultrasound research system us4R-lite™ (us4us Ltd., Poland) in a 64:256 channel configuration and standard angled 32-element Phased-Array probes. The system can acquire and store raw RF or demodulated I/Q data at a speed of 2–6 GB/s, enabling real-time FMC at high speed. Data can be stored on a PC during scanning and processed by a high-performance GPU. We have successfully tested our experimental setup while scanning flat-section welds with a motorized scanner at a speed approaching 100 mm/s. The acquisition and processing software developed uses Nvidia CUDA on GPU and can manage real-time storage and scanning. Next, we are planning to integrate the solution into an industrialgrade high-speed FMC acquisition system with embedded GPU processing.

Keywords:
Ultrasonic Testing (UT) (4285), robotic inspection (23), PAUT (42), FMC (16), TFM (28), GPU processing

Abstract:
In this paper, we propose a novel deep learning method for joint classification and segmentation of breast masses based on radio-frequency (RF) ultrasound (US) data. In comparison to commonly used classification and segmentation techniques, utilizing B-mode US images, we train the network with RF data (data before envelope detection and dynamic compression), which are considered to include more information on tissue’s physical properties than standard B-mode US images. Our multi-task network, based on the Y-Net architecture, can effectively process large matrices of RF data by mixing 1D and 2D convolutional filters. We use data collected from 273 breast masses to compare the performance of networks trained with RF data and US images. The multi-task model developed based on the RF data achieved good classification performance, with area under the receiver operating characteristic curve (AUC) of 0.90. The network based on the US images achieved AUC of 0.87. In the case of the segmentation, we obtained mean Dice scores of 0.64 and 0.60 for the approaches utilizing US images and RF data, respectively. Moreover, the interpretability of the networks was studied using class activation mapping technique and by filter weights visualizations.

Keywords:
breast mass classification, breast mass segmentation, convolutional neural networks, deep learning, quantitative ultrasound, ultrasound imagin

Abstract:
In this work, we propose a deep learning method for breast mass segmentation in ultrasound (US). Variations in breast mass size and image characteristics make the automatic segmentation difficult. To addressthis issue, we developed a selective kernel (SK) U-Net convolutional neural network. The aim of the SKswas to adjust network's receptive fields via an attention mechanism, and fuse feature maps extractedwith dilated and conventional convolutions. The proposed method was developed and evaluated usingUS images collected from 882 breast masses. Moreover, we used three datasets of US images collectedat different medical centers for testing (893 US images). On our test set of 150 US images, the SK-U-Netachieved mean Dice score of 0.826, and outperformed regular U-Net, Dice score of 0.778. When evaluatedon three separate datasets, the proposed method yielded mean Dice scores ranging from 0.646 to 0.780. Additional fine-tuning of our better-performing model with data collected at different centers improvedmean Dice scores by ~6%. SK-U-Net utilized both dilated and regular convolutions to process US images. We found strong correlation, Spearman's rank coefficient of 0.7, between the utilization of dilated convo-lutions and breast mass size in the case of network's expansion path. Our study shows the usefulness ofdeep learning methods for breast mass segmentation. SK-U-Net implementation and pre-trained weightscan be found at github.com/mbyr/bus_seg.

Keywords:
attention mechanism, breast mass segmentation, convolutional neural networks, deep learning, receptive field, ultrasound imaging

Abstract:
We propose a novel approach to breast mass classification based on deep learning models that utilize raw radio-frequency (RF) ultrasound (US) signals. US images, typically displayed by US scanners and used to develop computer-aided diagnosis systems, are reconstructed using raw RF data. However, information related to physical properties of tissues present in RF signals is partially lost due to the irreversible compression necessary to make raw data readable to the human eye. To utilize the information present in raw US data, we develop deep learning models that can automatically process small 2D patches of RF signals and their amplitude samples. We compare our approach with classification method based on the Nakagami parameter, a widely used quantitative US technique utilizing RF data amplitude samples. Our better performing deep learning model, trained using RF signals and their envelope samples, achieved good classification performance, with the area under the receiver attaining operating characteristic curve (AUC) and balanced accuracy of 0.772 and 0.710, respectively. The proposed method significantly outperformed the Nakagami parameter-based classifier, which achieved AUC and accuracy of 0.64 and 0.611, respectively. The developed deep learning models were used to generate parametric maps illustrating the level of mass malignancy. Our study presents the feasibility of using RF data for the development of deep learning breast mass classification models.

Keywords:
breast lesion classification, convolutional neural networks, deep learning, radio-frequency signals, ultrasound imaging

Abstract:
Accurate estimation of the speed-of-sound (SoS) is important for ultrasound (US) image reconstruction techniques and tissue characterization. Various approaches have been proposed to calculate SoS, ranging from tomography-inspired algorithms like CUTE to convolutional networks, and more recently, physics-informed optimization frameworks based on differentiable beamforming. In this work, we utilize implicit neural representations (INRs) for SoS estimation in US. INRs are a type of neural network architecture that encodes continuous functions, such as images or physical quantities, through the weights of a network. Implicit networks may overcome the current limitations of SoS estimation techniques, which mainly arise from the use of non-adaptable and oversimplified physical models of tissue. Moreover, convolutional networks for SoS estimation, usually trained using simulated data, often fail when applied to real tissues due to out-of-distribution and data-shift issues. In contrast, implicit networks do not require extensive training datasets since each implicit network is optimized for an individual data case. This adaptability makes them suitable for processing US data collected from varied tissues and across different imaging protocols. We evaluated the proposed SoS estimation method based on INRs using data collected from a tissue-mimicking phantom containing four cylindrical inclusions, with SoS values ranging from 1480 m/s to 1600 m/s. The inclusions were immersed in a material with an SoS value of 1540 m/s. In experiments, the proposed method achieved strong performance, clearly demonstrating the usefulness of implicit networks for quantitative US applications.

Keywords:
beamforming, deep learning, implicit neural representations, speed-of-sound, quantitative ultrasound

Abstract:
Background, Motivation and Objective
Flexible ultrasound (US) arrays are a promising technology that may further democratize US technology — e.g. in wearable US. Flexible transducers also pose challenges in image reconstruction, as they require adaptable beamforming delays due to a changing geometry of the probe. Various approaches have been proposed for flexible array shape estimation and beamforming, e.g. external sensors, deep learning and optimization. In this work, we propose a deep reinforcement learning (DRL) approach, where a software agent is responsible for tracking the array shape to properly reconstruct US B-mode image.

Statement of Contribution/Methods
Here we considered a reinforcement learning environment as a setup consisting of a US system with the flexible array and point targets phantom. The environment was simulated using j-Wave software. The environment’s state consisted of the current shape of the array, modeled as a sinusoid s = a sin(bx + c), and the current model of the array assumed by the beamformer: s’ = a’ sin(b’x + c’) (single-element STA scheme). A single episode consisted of
7 steps; the parameters a, b, c could vary from step to step (within bounds of physical constraints). The agent observed the current B-mode, could modify the current values of a’, b’ and c’ (action) and received a reward equal to the linear combination of the coherence factor and structural similarity index measure (SSIM) between the current and reference image. We trained our agent using TD3 approach and tested it for various settings of a, b and c.

Results/Discussion
Our agent achieved an average SSIM of 0.73 per episode step. Figure 1 shows the sequence of states and images within an example episode; the agent was able to correctly react to the change of the array shape. The DRL approach has the following advantages compared to other methods: the agent can be trained to operate in an environment with a changing state; and the agent can be trained to maximize expected return (dependent on beamforming quality metric), which does not have to be differentiable.

Fig. 1. (Top) B-mode images, estimated array shape (red) and actual shape (black) within an example episode. (Bottom) Coherence factor and SSIM achieved by the agent during the episode

Abstract:
Ultrasound (US) imaging is widely used for the tissue characterization. However, US images commonly suffer from speckle noise, which degrades perceived image quality. Various deep learning approaches have been proposed for US image denoising, but most of them lack the interpretability of how the network is processing the US image (black box problem). In this work, we utilize a deep reinforcement learning (RL) approach, the pixelRL, to US image denoising. The technique utilizes a set of easily interpretable and commonly used filtering operations applied in a pixel-wise manner. In RL, software agents act in an unknown environment and receive appropriate numerical rewards. In our case, each pixel of the input US image has an agent and state of the environment is the current US image. Agents iteratively denoise the US image by executing the following pixel-wise pre-defined actions: Gaussian, bilateral, median and box filtering, pixel value increment/decrement and no action. The proposed approach can be used to generate action maps depicting operations applied to process different parts of the US image. Agents were pre-trained on natural gray-scale images and evaluated on the breast mass US images. To enable the evaluation, we artificially corrupted the US images with noise. Compared with the reference (noise free US images), filtration of the images with the proposed method increased the average peak signal-to-noise ratio (PSNR) score from 14 dB to 26 dB and increased the structure similarity index score from 0.22 to 0.54. Our work confirms that it is feasible to use pixel-wise RL techniques for the US image denoising.

Keywords:
deep reinforcement learning, ultrasound imaging, image denoising, filtration, breast cancer

Abstract:
In this work, we considered the implementation of a 3D volume reconstruction algorithm for single plane-wave ultrasound insonification. We review the theory behind the Hybrid Spectral-Domain Imaging (HSDI) algorithm, provide details of the algorithm implementation for Nvidia CUDA GPU cards, and discuss the performance evaluation results. The average time required to reconstruct a single data volume using our GPU implementation of the HSDI algorithm was 22 ms. We also present an iso-surface extraction result using a marching cubes algorithm. Our work constitutes a preliminary research for further development and implementation of 3D volume reconstruction using GPU implementation of the spectral domain imaging algorithm.

Keywords:
ultrasound imaging, 3D ultrasound, volumetric imaging, gpu

Abstract:
Ultrasound is becoming the modality of choice for everyday medical diagnosis, due to its mobility and decreasing price. As the availability of ultrasound diagnostic devices for untrained users grows, appropriate guidance becomes desirable. This kind of support could be provided by a software agent, who easily adapts to new conditions, and whose role is to instruct the user on how to obtain optimal settings of the imaging system during an examination. In this work, we verified the feasibility of implementing and training such an agent for ultrasound, taking the deep reinforcement learning approach. The tasks it was given were to find the optimal position of the transducer’s focal point (FP task) and to find an appropriate scanning plane (PP task). The ultrasound environment consisted of a linear-array transducer acquiring information from a tissue phantom with cysts forming an object-of-interest (OOI). The environment was simulated in the Field-II software. The agent could perform the following actions: move the position of the probe to the left/right, move focal depth upwards/downwards, rotate the probe clockwise/counter-clockwise, or do not move. Additional noise was applied to the current probe setting. The only observations the agent received were B-mode frames. The agent acted according to stochastic policy modeled by a deep convolutional neural network, and was trained using the vanilla policy gradient update algorithm. After the training, the agent’s ability to accurately locate the position of the focal depth and scanning plane improved. Our preliminary results confirmed that deep reinforcement learning can be applied to the ultrasound environment.

Keywords:
ultrasound guidance, reinforcement learning, deep learning

Abstract:
Ultrasound medical diagnostics is a real-time modality based on a doctor's interpretation of images. So far, automated Computer-Aided Diagnostic tools were not widely applied to ultrasound imaging. The emerging methods in Artificial Intelligence, namely deep learning, gave rise to new applications in medical imaging modalities. The work's objective was to show the feasibility of implementing deep learning algorithms directly on a research scanner with GPU software beamforming. We have implemented and evaluated two deep neural network architectures as part of the signal processing pipeline on the ultrasound research platform USPlatform (us4us Ltd., Poland). The USPlatform is equipped with a GPU cluster, enabling full software-based channel data processing as well as the integration of open source Deep Learning frameworks. The first neural model (S-4-2) is a classical convolutional network for one-class classification of baby body parts. We propose a simple 6-layer network for this task. The model was trained and evaluated on a dataset consisting of 786 ultrasound images of a fetal training phantom. The second model (Gu-net) is a fully convolutional neural network for brachial plexus localisation. The model uses 'U-net'-like architecture to compute the overall probability of target detection and the probability mask of possible target locations. The model was trained and evaluated on 5640 ultrasound B-mode frames. Both training and inference were performed on a multi-GPU (Nvidia Titan X) cluster integrated with the platform. As performance metrics we used: accuracy as a percentage of correct answers in classification, dice coefficient for object detection, and mean and std. dev. of a model's response time. The 'S-4-2' model achieved 96% classification accuracy and a response time of 3 ms (334 predictions/s). This simple model makes accurate predictions in a short time. The 'Gu-net' model achieved a 0.64 dice coefficient for object detection and a 76% target's presence classification accuracy with a response time of 15 ms (65 predictions/s). The brachial plexus detection task is more challenging and requires more effort to find the right solution. The results show that deep learning methods can be successfully applied to ultrasound image analysis and integrated on a single advanced research platform

Keywords:
Ultrasonic imaging, Neural networks, Convolution, Machine learning, Image segmentation, Kernel

Abstract:
Ultrasound Computer Tomography (USCT) enables
imaging of the spatial distribution of physical parameters, such as speed of sound and acoustic attenuation. These parameters are highly relevant in the context of tissue diagnostics, as pathological regions, for example tumors, often exhibit distinct attenuation and speed-of-sound characteristics compared to healthy tissue.Achieving USCT imaging at a clinically reasonable frame rate presents a number of challenges. The frame rate is constrained by several factors: the time of flight of ultrasound pulses between the source and the receivers, the number of transmit/receive events, the transfer of acquired data to the host computer, and
the time required for data processing and image reconstruction.
Among these, the physical limitations imposed by the time of
flight—necessary for a pulse to fully traverse the imaged ob-
ject—are particularly critical. While data transfer and processing times may be reduced through hardware acceleration and soft-ware optimization, these fundamental physical constraints cannot be circumvented in the same manner. Consequently, the reduction of the number of transmissions remains the only practical strategy to decrease this component of the total acquisition time.Although such a reduction generally results in some deterioration of image quality, this degradation can be mitigated through interpolation. This strategy shortens both the acquisition and data transfer stages, while shifting the computational burden to more intensive data processing. The objective of the present study is to demonstrate a proof of concept for this approach in USCT imaging, employing a neural network–based interpolator to restore image quality.

Abstract:
Various methods for reconstruction of atomistic models of dislocations and stacking faults in crystalline heterostructures are discussed. In molecular dynamics, for example, the atomistic models of dislocations are often obtained by means of elastic-plastic relaxation of a perfect crystal lattice subjected to external loading. Such a method does not give the possibility for emerging atomistic models of an arbitrarily chosen network of dislocations. This problem concerns many sets of dislocations observed by means of the high-resolution transmission electron microscopy (HRTEM). In this presentation we discuss a deterministic method for obtaining atomistic models of dislocations. The method is based on the sequential inserting of single dislocations into the crystal lattice. In this case, the analytic formulas for gliding of single dislocations in elastic continuum are used. The method is based on the symbolic algebra of elemental lattice distortion tensor fields. Contrary to the linear strain and linear rotation measures, the lattice distortion tensor is the correct measure of finite deformation. Thus, on the basis of lattice distortion tensor field, many different tensor fields of finite strain and finite rotation of the elements of crystal lattice can be determined uniquely. Among others, this enables generation of atomistic models in terms of finite deformation approach.

The method presented here links: (i) the analytic formulas for lattice distortion tensor fields derived from the linear theoryof dislocations, (ii) the finite deformation algebra of distortion fields, and (iii) the atom-by-atom reconstruction of dislocations including their core structures. This method has been implemented in a visual editor of dislocations. Configurations of atoms obtained in this way satisfy the stress equilibrium equations in terms of the linear elasticity. On the other hand, the spatial Burgers vectors of dislocations are stretched and rotated to each other according to the finite deformation theory. During generation of atomistic models, such reconstructed crystal lattice is intersected by the discontinuities corresponding to the insertions of subsequent dislocations along deformed slip planes.

Independently of the atomistic models, a method for generation of finite element (FE) models being the continuous counterpart of the atomistic ones is also discussed. The finite element models discussed are based on the use of the continuous distortion tensor field extracted from 2D or 3D HRTEM images of dislocations. In result, two models are obtained: FE model and the atomistic one which can be embedded in the FE mesh. Such generated atomistic and FE meshes of defects are used as the input data for modelling by means of the ab-initio method, molecular dynamics, statics, and other atomistic methods.

An example of a WYSIWYG editor for generation of atomistic and FE meshes is presented. The program, the Visual Editor of Crystal Defects, has been written with the use of C++, Qt4 and OpenGL. Its successor, Wurtzite, written in Python, is also presented in brief.

Keywords:
dislocations, editor of crystal defects, distortion tensor fields, algebra of dislocations

Abstract:
Atomistic models of dislocations in crystalline structures are often obtained by means of elastic-
plastic relaxation of a perfect crystal lattice subjected to external loading. Another method is based
on inserting of single dislocations into the perfect lattice. In this case the analytic formulas for the
glide of a single dislocation in elastic continuum are used. The methods mentioned above do not
give the possibility for emerging atomistic model of an arbitrary chosen network of dislocations. This
problem concerns many sets of dislocations observed by means of high resolution transmission
electron microscopy. In this presentation we introduce a deterministic method for obtaining atomistic
models of dislocations. The method is based on the use of symbolic algebra of elemental lattice
distortion tensor fields. Contrary to the linear strain and rotation measures, the lattice distortion
tensor is the correct measure of finite deformation. Thus, on the basis of distortion field, many
different tensor fields of finite strains and rotations can be determined uniquely. This enables
generation of atomistic models in terms of finite deformation approach [1,2]. The method presented
here links: (i) the analytic formulas for lattice distortions derived from the linear theory of dislocations,
(ii) the finite deformation algebra of distortion fields, and (iii) the atom-by-atom reconstruction of
dislocations including their core structures. This method has been implemented in a visual editor of
dislocations. Configurations of atoms obtained in this way satisfy the stress equilibrium equations in
terms of linear elasticity. On the other hand, the spatial Burgers vectors of dislocations are stretched
and rotated to each other according to the finite deformation theory. The resultant net of atoms can
used as input data to ab-initio and/or molecular dynamics programs to find a low energy configuration
corresponding to a given interatomic potential.

[1] Łażewski J., Jochym P.T., Piekarz P., Sternik M., Parlinski K., Cholewiński J., Dłużewski P.,
Krukowski S., J. Mater. Sci. 54, 10737-10745, 2019.
[2] Cholewiński J., Maździarz M., Jurczak G., Dłużewski P., Int. J. Multiscale Comp. Eng. 9, 411-421,
2014.

Abstract:
In this work, we present a low-cost, portable, and fully configurable ultrasound system implementing 2-D real-time Shear Wave Elastography (SWE) imaging mode. To achieve that we have enhanced the transmit capabilities of the 256 TX/64 RX us4R-lite research system, developed by our team, to support push pulses generation. This system was combined with a signal processing pipeline reconstructing stiffness maps from raw RF data. Real-time imaging performance was provided by an efficient reconstruction algorithm execution that incorporated graphics processing unit (GPU). The overall system performance was assessed experimentally using an industry-standard elasticity Q/A phantom. Relevant reconstruction parameters were evaluated in terms of reconstruction time. The system achieved stiffness estimation with a bias <5% and SNR of 30 dB and was able to detect lesions of size >4 mm and various stiffness with CNR in the range of 13–17 dB. The system throughput of up to 5 fps has been achieved on a PC notebook equipped with NVIDIA RTX 3060 GPU.

Keywords:
Ab-initio, atomistic modeling, tensor fields, dislocation fields algebra, lattice distortions, visualization methods

Abstract:
The uNET neural network architecture has shown very promising results when applied to semantic segmentation of biomedical images. The aim of this work is to check whether this architecture is equally applicable to semantic segmentation distinguishing the structural elements of railway viaducts. Artificial images generated by a computer graphics program rendering the 3D model of the viaduct in a photorealistic manner will be used as data sets. This approach produces a large number of
images that provide a solid training set for machine learning model.

Keywords:
Computer vision, deep learning, semantic segmentation

Abstract:
The ultimate goal of this work is a real-time processing framework for ultrasound image reconstruction augmented with machine learning. To attain this, we have implemented WaveFlow – a set of ultrasound data acquisition and processing tools for TensorFlow. WaveFlow includes: ultrasound Environments (connection points between the input raw ultrasound data source and TensorFlow) and signal processing Operators (ops) library. Raw data can be processed in real-time using algorithms available both in TensorFlow and WaveFlow. Currently, WaveFlow provides ops for B-mode image econstruction (beamforming), signal processing and quantitative ultrasound. The ops were implemented both for the CPU and GPU, as well as for built-in automated tests and benchmarks. To demonstrate WaveFlow’s performance, ultrasound data were acquired from wire and cyst phantoms and elaborated using selected sequences of the ops. We implemented and valuated: Delay-and-Sum beamformer, synthetic transmit aperture imaging (STAI), planewave imaging (PWI), envelope detection algorithm and dynamic range clipping. The benchmarks were executed on the NVidiaR Titan X GPU integrated in the USPlatform research scanner (us4us Ltd., Poland). We achieved B-mode image reconstruction frame rates of 55 fps, 17 fps for the STAI and the PWI algorithms, respectively. The results showed the feasibility of realtime ultrasound image reconstruction using WaveFlow operatorsin the TensorFlow framework. WaveFlow source code can be found at github.com/waveflow-team/waveflow.