Knowledge assisted machine learning to clarify pore influence on fatigue life of forging/additive hybrid manufactured Ti-17 alloy

Specimen and fatigue test

Ti-17 segments were produced with WAAMed layers on a forged base [Figure 1A]. Standard bar samples of ϕ4 mm [Figure 1B] were taken from the hybrid regions and the rotating-bending fatigue tests were performed at ambient temperature in line with the GB/T 4337-2008. The effective stress at the narrowest point of the specimen was set as 500 MPa. The samples rotated at a speed of 5,000 r/min with a stress ratio R = -1.

Figure 1. (A) Forging/additive hybrid manufactured Ti-17 part and the microstructures of the additively manufactured and forged parts; (B) Schematic of the rotating bar bending fatigue sample (unit in mm). Ti-17: Ti-5Al-2Sn-2Zr-4Mo-4Cr.

Theoretical models

Back propagation neural network

The accuracy of the back propagation neural network prediction results is strongly influenced by the training process, as they are based on maximum likelihood algorithms inspired by neural networks in the brain. In engineering, back propagation neural networks are widely used to solve complex problems because of their faster and more accurate predictive power. In these networks, the mathematical equations for how inputs and outputs map together can be learned through training to get the best possible result. The schematic diagram of the back propagation neural network used is shown in Figure 2A.

Figure 2. (A) Back propagation neural network model structure diagram; (B) GAN model structure diagram. GAN: Generative adversarial network.

The first and last layers of a neural network are called the input and output layers, respectively, while the middle layer is called the hidden layer. A neural network is trained by passing signals between the layers. The operation of a single neuron can be expressed as follows:

(1) $$ y_{i}=f_{i}\left(\sum w_{i j} x_{j}-t_{i}\right) $$

where y_i represents the output, w_ij is the weights, x_j is the inputs, t_i is the inputs and f_i is the activation function.

Sigmoid, Relu and Tanh functions are some of the commonly used activation functions^[39,40]. With gradient descent, the back propagation neural network uses error back propagation to train its Multilayer feed-forward network. It uses mathematical chain rules to calculate the gradient of each layer, and then returns to the input layer to adjust the weights. This algorithm calculates the error rate of the predicted data^[41]. As the network’s actual output value compares with its desired output value, this step is repeated until the mean square error (MSE) is as small as possible. The coefficient of determination (R²) and MSE, which are used to assess the prediction accuracy, are expressed as

(2) $$ R^{2}=\frac{\sum_{n}^{i=1}\left(\hat{y}_{i}-\hat{y}_{i}\right)^{2}}{\sum_{n}^{i=1}\left(y_{i}-\hat{y}_{i}\right)^{2}} $$

(3) $$ \text { MSE }=\frac{1}{n} \sum_{n}^{i=1}\left(y_{i}-\hat{y}_{i}\right)^{2} $$

where $$ \hat{y}_i $$ is the predicted value, and the rest of the parameters are the same as above.

The input variables are the defect size (pore size) and the distance to surface (pore location), while the output variable is the fatigue life. Using a random distribution, 70% of the dataset was employed for training, while 15% was allocated for validation and 15% for testing. The reference or hidden layer of a back propagation neural network model is generally determined without a uniform rule. This work uses back propagation neural networks with a topology of three, four and five neurons in the first hidden layer [Table 1]. According to the results, when the number of neurons in the first hidden layer is 4, the model has the best overall performance.

Table 1

Predicted performance of the back propagation neural network models with different numbers of neurons in the first hidden layer

Back propagation neural network	Number of neurons	Metrics
	1st layer	R2			MSE
		Training	Validation	Testing	Training	Validation	Testing
Model 1	3	0.835997	0.829817	0.837631	0.088284	0.657599	0.249147
Model 2	4	0.804974	0.098125	0.944086	0.161155	0.095284	0.275916
Model 3	5	0.807096	0.920224	0.921055	0.201762	0.210129	0.155929

R²: The coefficient of determination; MSE: mean square error.

Physical knowledge model

The general relationship between the fatigue stress and the fatigue life^[42-46], i.e., the S-N curve, is written as

(4) $$ \sigma _{eff}\ast N_f=C $$

In additive manufactured materials^[47-50], the existence of pores produces additional stress concentration, which largely influence the fatigue performance^[51-53]. Regarding the relationship between fatigue life and defect parameters^[5,54,55], previous investigations have derived

(5) $$ \left \{ S\sigma_a(area_{eff})^{1/12}L^{\beta} \right \} ^{\alpha}N_f=C $$

(6) $$ L=1-\frac{d_1+d_2}{2r} $$

where N_f refers to the fatigue life, with C as a constant, S the defect shape factor, σ_a the unified stress amplitude, area_eff the effective size of the defects, α and β the material dependent constants, r the radius of the fatigue fracture surface, and d₁ and d₂ the distances from defect to fatigue fracture surface. However, in rotating-bending fatigue tests, since the overall stress is related to the distance to sample surface, the above equations do not apply and the parameter of the distance of defects to the sample surface needs to be considered. In this study, the pore size ranges between 10 to 250 μm, and under the rotating-bending condition, stress concentration in the whole sample needs to be considered. We thus employ finite element analysis to evaluate stress concentration and reemploy the common fatigue model of Equation (4) in the subsequent model development.

We first evaluate the influence of pore size and location on stress distribution in the samples by simulating the bending of pore-free and pore-containing samples with different pore diameter and locations with the finite element method (FEM) employing the software ANSYS. Tetrahedral mesh partitioning method is used to build a mesh model for the three-dimensional solid model and mesh size is 0.05 mm for all parts. The minimum time step was set to 10^-5 s. The left end of the sample was fixed and a 500 N torsional force was applied along the Y direction at the right end [Figure 3].

Figure 3. Boundary conditions for finite element simulation.

The equivalent stress distribution on the sample cross sections is systematically studied and Figure 4A-D exemplifies the cases for near-surface [Figure 4A], half-radius [Figure 4B], center [Figure 4C] pores of 100 μm in radius and a pore-free case [Figure 4D]. To evaluate the stress concentration, the overall stress ratio at a fixed point P (0.05 mm from the surface) with respect to pore size and location is shown in Figure 4E and F. In general, by fitting the FEM results, the stress ratio increases with the pore size and decreases with pore distance to surface with the following relationship.

Figure 4. Equivalent stress distribution on the cross sections of the pore-containing (A-C) and pore-free (D) samples, pores are 100 um in radius. Equivalent stress distribution of pore size and location on the Y axis negative direction from the edge to the center of the circle at 0.05 mm from the fixed point; (E) black dots are equivalent stress and (F) data were fitted using Equation (7). Blue dots are finite element simulation data, red dots are equation fitting data.

(7) $$ \frac{\sigma_{E-pore-contain}}{\sigma_{E-pore-free}}=\frac{S* 1.005}{1+x^{1.1291}} * 1.2295^{1.0172}+0.8868 $$

where S is the pore size and x is the pore location.

Regarding the above stress concentration ratio and considering the potential size and location limit in this work, the effective stress after can be written as:

(8) $$ \sigma_{e f f}=\sigma_{0} *\left[1+\frac{a *(\frac{S}{S_{0}})^{b}}{\left(1+\frac{x}{x_{0}}\right)^{c}}\right]^{m} $$

where σ₀ is the fatigue stress during the test (500 MPa in the present work), with a, b, c, m, S₀, and x₀ being constants.

The relationship between σ_eff and fatigue life (N_f) is:

(9) $$ \sigma_{e f f} * N_{f}=\sigma_{0} *\left[1+\frac{a *\left(\frac{S}{S_{0}}\right)^{b}}{\left(1+\frac{x}{x_{0}}\right)^{c}}\right]^{m} * N_{f}=I $$

(10) $$ \log N_{f}=\log I-\log \sigma_{0}-\log \left[1+\frac{a *(\frac{S}{S_0})^b}{\left(1+\frac{x}{x_{0}}\right)^{c}}\right]^{m}=d-m * \log \left[1+\frac{a *(\frac{S}{S_{0}})^{b}}{\left(1+\frac{x}{x_{0}}\right)^c}\right] $$

where d is the material constants, and the rest of the parameters are the same as above.

Overall strategy

In this work [Figure 5], 34 sets of defect size, defect to distance and fatigue life data were obtained from fracture photos after forging/additive components experienced rotating-bending fatigue fracture. The sparse and limited data set then prevented further analysis of the work. In order to overcome the influence of data quality, the corresponding fatigue life equation is established on the basis of physical knowledge, and it is applied to data processing and model interpretation. In order to overcome the influence of data quantity, a GAN based on Pytorch deep learning framework is constructed that conforms to the laws of defect size, distance to surface and fatigue life. Finally, a machine learning framework with physical knowledge predicts the fatigue life to be smooth by defect size and defect to distance.

Figure 5. The overall flow chart of the present work. A total of 34 sets of defect size, defect to distance and fatigue life data were obtained from fracture photos after forging/additive components experienced rotating-bending fatigue fracture. The sparse and limited data set then prevented further analysis of the work. In order to overcome the influence of data quality, the corresponding fatigue life equation is established on the basis of physical knowledge, and it is applied to data processing and model interpretation. In order to overcome the influence of data quantity, a GAN based on Pytorch deep learning framework is constructed that conforms to the laws of defect size, distance to surface and fatigue life. Finally, a machine learning framework with physical knowledge predicts the fatigue life to be smooth by defect size and defect to distance.

Fracture morphology

The fracture morphology was observed under scanning electron microscopy for spherical crack sources. Typical fractographs are shown in Figure 6 with the fatigue cycles to failure and defect characters. Generally, samples with larger defects and shorter distance to surface exhibit shorter fatigue life. The sample with a pore size of Φ127.5 μm closest to the surface displays the shortest life of 29,700 cycles [Figure 6A]. Samples with intermediate pore size and distance to surface demonstrate intermediate lives of 254,000 [Figure 6B] and 545,000 cycles [Figure 6C]. The sample with the smallest pore size and largest distance to surface presents the longest fatigue life of 6,410,000 cycles [Figure 6D].

Figure 6. Fatigue fractographs with the size and location of the spherical pores after cycles of (A) 29,700, (B) 254,000, (C) 561,000 and (D) 6,410,000.

Data preprocessing

Owing to the existence of pores, a majority of the fatigue samples fail at the hybrid regions. Due to the lack of data for arbitrarily shaped pores, only that from spherical pores is collected. The original 34 sets of experimental data with the fatigue life against the pore size and location are shown in Figure 7A and B. For data preprocessing, we have operations such as logging the fatigue life. We find that the fatigue life processed by log can better find the relationship with the defect parameters [Figure 7C and D]. However, remarkable sparsity is readily observed, which hinders the immediate employment of machine learning algorithms. Therefore, preprocessing of the raw data is necessary for the successful and accurate prediction of fatigue life.

Figure 7. (A and B) Relationship between fatigue life with and without log treatment and defect parameters. The experimental data with the fatigue life of the bending samples against (C) the size and (D) location of the spherical pores.

Then, outlier rejection was performed using statistical methods, employing two distinct approaches. Values that do not fall within the interval, i.e., mean ± 2× standard deviation, are considered outliers. Outliers are defined as values that are not within the interval, i.e., lower four medians -1.5× median deviation, upper four medians +1.5× median deviation. Figure 8A and B shows the box line plots of defect size, distance to surface and fatigue life. After discrete value rejection of the raw data, the data quality has been greatly improved. Finally, the data of defect size, distance to surface and fatigue life are in the vicinity of 50, 100 and 5.8 intervals, respectively. Figure 8C and D shows the heatmaps of defect size, distance to surface and fatigue life from the raw data and the data after discrete value rejection. With such data processing, the relationship between the independent and dependent variables gradually improves and is more conducive to subsequent model training. After preprocessing, there are 30 groups of data.

Figure 8. Box line plots of the independent and dependent variables. (A) Raw data (B) Data after discrete value rejection. Heatmaps of defect size, distance to surface and fatigue life. (C) Raw data (D) Data after discrete value rejection.

Of course, operations such as feature selection, data cleaning, cross-validation and splitting and their specific effects on the model are described in turn in Section “Model performance and remarks”, in Section “Data filtering” and in Section “Fatigue life model”. It is important to note that these operations are formally different from data processing in regular machine learning models, and we have incorporated them into every step of the process in this work.

Data filtering

When Equation (10) was fitted to the initial datasets with S₀ = 100 and x₀ = 2,000 (the sample radius), the resulting R² was found to be 0.5527 with d = 6.7106, m = 0.8087, a = 185.1749, b = 3.3880 and c = 15.2355. The unacceptably small R² indicates the deviation of certain data points from the physical knowledge model, which leads to catastrophic results during the fitting and implies the necessity of data filtering. To avoid arbitrariness, we combine d, m, a, b and c parameters in the range of 0.1, so as to reasonably select the data with the largest deviation to all possible models. The parameter combination of d, m, a, b, c and corresponding R² for the 243 physical knowledge models are listed in Table 2.

For example, the parameter combination is performed with the step interval of 0.1. With d = 6.6106/6.7106/6.8106, m = 0.7087/0.8087/0.9087, a = 185.0749/185.1749/185.2749, b = 3.2880/3.3880/3.4880 and c = 15.1355/15.2355/15.3355, a total of 243 combinations of d, m, a, b, c and corresponding R² are obtained.

Generally, R² of 0.5-0.6 is reached after the 243 physical knowledge models [Table 2], but still far from satisfactory. Figure 9 shows the experimental data and one of the fitted surfaces with d = 6.7106, m = 0.8087, a = 185.1749, b = 3.3880 and c = 15.2355 with R² = 0.5527. Remarkable deviation from the fitted models exists for a number of data points. We thus gathered the mean absolute percentage deviation (MAPE) of the 30 groups of experimental data to all 243 physics knowledge models. The MAPE is defined according to

Figure 9. The fatigue life against the defect size and distance to surface for the experimental data (red dots) and one of the fitted surfaces with d = 6.7106, m = 0.8087, a = 185.1749, b = 3.3880 and c = 15.2355 with R² = 0.5527. R²: The coefficient of determination.

where n is the total number of data, and y_i and $$ \hat{y}_i $$ the actual and fitted values of the ith data, respectively. The MAPE for each group of data is shown in Figure 10. A general criterion of 10% is set to filter the data and groups 11, 24, 25, 30 are thus filtered. However, despite the relatively large deviation of data group 2 (10.5%), it is retained due to relatively small deviation on certain models. After filtering, there are 26 groups of data.

Figure 10. The overall deviation of the 30 groups of experimental data to all 243 physics knowledge models.

Data augmentation

Subsequently, a GAN is built. The GAN learns to generate values that match patterns in the defect size, the distance to surface, and the fatigue life. The generator, a neural network with three output values, is trained to provide data that matches the defect size, the distance to surface, and the fatigue life pattern. On the other hand, the discriminator tries to determine whether it is from the real data source or from the generator, based on these three values. The steps for constructing and training the GAN in this paper are as follows:

(1) Previewing data from a real dataset; (2) Determining whether the discriminator can at least distinguish between actual data and random noise; (3) Determining whether an untrained generator can produce data in the required format; (4) Displaying the observed loss values to understand the training progress; (5) Producing fresh data; (6) Repeat the data filtering process.

The loss values of the GAN, the discriminator and the generator are shown in Figure 11A-C, respectively. When the discriminator is ineffective at telling the authentic data from the fake data, it cannot decide whether to output 0 or 1, and instead outputs 0.5, and the loss value is 0.5² = 0.25. The GAN advances gradually as the training goes on since the loss value reduces gradually and perceivably. The loss value increases again to 0.25 later in the training. This indicates that the generator has mastered the art of producing data in a predictable fashion, rendering the discriminator ineffective. In other words, the discriminator’s output is 0.5 and the loss value jumps to 0.25. Note that due to the initially limited number of experiment data with large sparsity, GAN was able to produce only two qualified groups of data. After augmentation, there are 28 groups of data.

Figure 11. Loss value. (A) GAN; (B) Discriminator; (C) Generator. GAN: Generative adversarial network.

Fatigue life model

Finally, after data filtering and augmentation [Figure 12], the prediction surface becomes increasingly smooth. A total of 28 groups of data were employed to predict the fatigue life with only the back propagation neural network, as shown in Figure 13A and B, with a confidence interval of nearly 90% in Figure 13C. The predicted surface is close to the physical knowledge model (e.g., Figure 9) with the fatigue life decreasing with the defect size and increasing with the distance to surface. However, without physical knowledge, the resulting model is overfitted without interpretability.

Figure 12. In the process of model selection, the progress of the model is reflected in an increasingly smooth prediction surface.

Figure 13. Third prediction of experimental data after augmentation of GAN model using back propagation neural network. (A) 3D surface; (B) Contour map; (C) 2D confidence interval. The final physical model optimized with the lsqcurvefit function in MATLAB. (D) 3D surface; (E) Contour map; (F) 2D confidence interval. GAN: Generative adversarial network.

In order to achieve a precise model with physical knowledge, the 28 groups of data are again employed together with the physical knowledge models in Section “Physical knowledge model”. The lsqcurvefit function in MATLAB is utilized and the optimal constants are obtained as: d = 13.1254, m = 8.5685, a = 7.6417, b = 0.2401, c = 1.3565, S₀ = 51.0409 and x₀ = 564.6523 [Equation (12)]. The final fatigue life model after Equation (10) is as follows, which is visualized in Figure 13D-F.

The present model thus indicates the influence of the two main factors, i.e., pore radius and location, on the fatigue life. At a fixed pore location, the fatigue life decreases with pore radii, while at a fix pore radius, the fatigue life decreases with pore distance to surface. In particular, pores located near the surface will be detrimental to fatigue life even if they are very small, while pores located at the central axis of the sample will be remarkably safer. More subtle relationship between S and x can thus be evaluated under the circumstance of rotating-bending fatigue. With N_f ≥ 10⁷ as a criterion, the following expression holds,

which thus indicates the mutual relationship between x and S; i.e., to have a satisfactory fatigue life, the pores should be smaller as their location approaches the surface. Specifically, for near-surface pores with x → 0, only very small pores with S ~ 4 μm are allowed without significant reducing fatigue life, while for typical pores under current additive manufacturing techniques with S ~ 1,000 μm, they can be located at least ~ 1,000 μm from the surface.

Model performance and remarks

We have also compared the methods used and the results obtained with the existing work. Since machine learning is a data-driven algorithm, the data itself has a significant effect on the model performance. Therefore, researchers have used different methods to prepare data before training a model. Li et al. developed a database for machine learning models using the Monte Carlo method^[56]. Zhan et al.^[57,58] proposed a continuous damage mechanics^[59,60] model that also incorporates the additive manufacturing effect with theoretical calculations for data augmentation. It was estimated that the R² for the training, validation and test sets are 0.9854, 0.9857 and 0.9854, respectively, with the overall R² = 0.9855. Nevertheless, it is unclear whether the high dispersion of fatigue data leads to a localized distribution of data, and whether the so-called theoretical models consider all the factors that influence fatigue data. In this paper, we propose a completely new approach for processing data combining GAN and physical knowledge models. The R² values for the training, validation and test sets are 0.8338, 0.9975 and 0.8912, respectively, while the total R² = 0.8701. A well-developed theoretical system and novel ideas underpin the GAN in deep learning. To overcome the limitations of localization of traditional methods, a continuous game of generative and discriminative networks can be used to learn the characteristics of data distribution.

The most essential activity in the model, i.e., feature engineering, influences the model performance. To estimate the fatigue life, Bao et al.^[13] have used ten sets of experimental data with three independent variables of defect size, defect location and defect type appearance, while Li et al.^[56] have used 22 sets of experimental data with five independent variables, i.e., flaw depth, defect size, defect-to-surface distance, maximum stress and construction direction. It can be shown that the geometric parameters of the critical flaws have a considerable influence on the fatigue life. As a result, in this paper, the fatigue life is predicted using two independent variables: defect size and distance to surface. We also provide a physical knowledge method for data processing based on these two variables. Lastly, it is worth mentioning that the present work has the largest data to variable ratio, i.e., 34 to 2, among the published work.

It is worth mentioning that, first, the present model only considers spherical pores, while in the hybrid manufacture samples, pores with other shapes also exist; second, the present model has not explicitly considered microstructure features, while pores located at grain/phase boundaries may have detrimental influence on fatigue life; third, the present model omits other metallurgical defects, e.g., lack of fusion, microcrack, etc., which is supposed to distinguish with appropriate additive manufacturing process and subsequent post-processing. While the above issues are covered in ongoing investigations, the results will be reported in forthcoming publications.