Over the past decade, interest in optimization has surged due to its pivotal role in addressing complex challenges across a range of fields, including computer science, data science, and engineering design¹. Optimization problems in these areas often present intricate structures with numerous decision variables, multiple constraints, and various objectives. As these fields advance, modern research increasingly incorporates sophisticated optimization algorithms to achieve efficient solutions under real-world limitations ². This trend is especially prominent in manufacturing ³, resource allocation ⁴, path planning ⁵, and financial portfolio management ⁶Given the demands for cost control, resource efficiency, and technological innovation, developing robust optimization techniques has become essential to meeting contemporary needs and driving further industry advancements.

In engineering, optimization approaches are frequently classified as analytical or numerical, with heuristic and metaheuristic methods addressing many sorts of complicated real-world issues ¹. For instance, in ⁷, a physics-informed neural network (HOMS-PINN) tackles elastic problems in composite materials, showcasing advanced numerical methods for multi-scale engineering challenges. Metaheuristic algorithms (MAs), such as those in ⁸, where a Snow Ablation Optimizer (SAO) optimizes PV systems, is commonly classified as a single-solution and population-based technique. Single-solution-based algorithms are more efficient, although population-based approaches, like the hybrid Grasshopper Optimization Algorithm (HGOA) in ⁹, excel in broader engineering optimization tasks. Many real-world issues are classed as single-objective, multi-objective, or multimodal, and these complexities frequently make classic mathematical methods, such as conjugate gradient and gradient descent, less successful ^{10 11}. In ¹², a multi-strategy Slime Mould Algorithm (MSMA) addresses engineering design problems, such as welded beam design, highlighting metaheuristics’ adaptability. Deterministic methods struggle with nonlinear, non-convex, or NP-hard problems, necessitating stochastic metaheuristics ¹³. For instance, in ¹⁴, an enhanced Slime Mould Algorithm (AGSMA) improves convergence for engineering applications, while ¹⁵ and ¹⁶ introduce Q-learning-enhanced JAYA (QLJAYA) and a stable Social Learning Swarm Optimizer (ESLPSO), respectively, for PV parameter estimation, demonstrating metaheuristics’ effectiveness in complex engineering scenarios.

Metaheuristic optimization algorithms have become a popular, cost-effective, and versatile solution to tackle complex optimization challenges. With their simplicity and ability to deliver optimal or near-optimal solutions quickly, they are highly effective for real-world applications ^17,18. These algorithms generally operate in two phases: exploration and exploitation. During exploration, the algorithm thoroughly searches across a feasible domain, while exploitation focuses on refining the search within promising regions. Balancing these two phases is essential to avoid local optima and to discover global solutions efficiently.

Metaheuristic algorithms are further divided into population-based and single-solution-based approaches ¹⁹. While single-solution algorithms concentrate on a single solution, population-based methods engage multiple solutions, allowing for robust exploration of the search space and enhanced avoidance of local optima. In single-solution-based algorithms, the search process focuses on intensifying a single solution. Examples include Simulated Annealing ²⁰, variable neighborhood search ²¹, iterated local search ²² and Tabu Search ²³, where each iteration modifies the current solution to move toward an optimal or near-optimal solution. These algorithms allow easy implementation and require only a small number of function evaluations during optimization. However, the disadvantages are the high possibility of trapping in local optima and failure to exchange information because these methods have only a single trend²⁴.

Population-based metaheuristics work by maintaining a population of potential solutions and exploring and improving them iteratively. These methods are commonly categorized based on their inspiration or methodological approach. Numerous classifications exist, but in this work, we adopt the classification used by²⁵ and ²⁶. These subcategories include evolution-based algorithms, Swarm Optimization algorithms, physics/chemistry-based algorithms, mathematics-based algorithms, and miscellaneous. Figure 1 gives the classification used in our work. Evolutionary algorithms draw inspiration from natural evolutionary processes and utilize operators modeled after biological phenomena, including crossover and mutation. The genetic algorithm (GA)²⁷, one of the most well-known evolutionary algorithms, is specifically influenced by Darwin’s theory of evolution. Swarm intelligence algorithms represent another category of meta-heuristics, inspired by the collective behaviors of animals when moving or hunting in groups. Traditional methods within this category include Particle Swarm Optimization ²⁸ and coot Optimization algorithm ²⁹. Physics and chemistry-based algorithms, such as the RIME algorithm ³⁰ and Atomic Orbit Search³¹, draw from physical and chemical principles to guide the search process. Mathematics-based algorithms use mathematical formulas and concepts, like trigonometric functions in the triangulation topology aggregation optimizer ³², improved Rung Kutta algorithm ³³, circle search algorithm, Exponential distribution optimizer (EDO)³⁴, Arithmetic Optimization Algorithm (AOA) ³⁵, etc, to navigate the optimization landscape. Miscellaneous Algorithms in our classification include algorithms inspired by human cognition and social behaviors (e.g., human-based metaheuristics) and strategies derived from game theory, competitive games (e.g., game-based metaheuristics), and other algorithms ²⁶. Human-based optimization metaheuristics such as mountaineering team-based optimization³⁶, the mother optimization algorithm (MOA)³⁷, the literature research optimization algorithm³⁸, the deep sleep optimizer³⁹, the offensive Defensive Optimization⁴⁰, Paper Publishing Based Optimization⁴¹ and etc. while, football game-based optimization ⁴², Billiards Optimization Algorithm⁴³, the puzzle optimization algorithm ⁴⁴, shell game optimization ⁴⁵, ring toss game-based optimization ⁴⁶, the golf optimization algorithm ⁴⁷, are examples of game-based metaheuristics. The following section will provide a detailed exploration of these algorithms, with particular emphasis on mathematics-based metaheuristics.

The diversity of these approaches highlights the adaptability of metaheuristics, as they can be tailored to suit different problem types and hybridized with other techniques to tackle increasingly complex optimization challenges. This flexibility and innovation make metaheuristics valuable tools across numerous fields, from engineering to artificial intelligence. Researchers have enhanced these algorithms with advanced concepts like hybridization, quantum computing, and chaotic maps to better address the complexities of modern optimization problems, making them indispensable tools for finding near-optimal solutions across a vast array of applications ^{48,49 50}.

Despite the abundance of established optimization algorithms, there remains a strong need for new, innovative approaches tailored to specific problem domains. The "No Free Lunch" theorem ⁵¹ underscores this necessity by stating that no single optimization algorithm is universally optimal for all problem classes. Each algorithm has strengths that make it suitable for certain types of problems, while it may perform less effectively in others. This insight has led to a continuous exploration of novel methods, aiming to enhance solution accuracy, convergence speed, and the balance between exploration and exploitation in search processes. In this light, the proposed algorithm is grounded in the principles of the classical derivative-based secant method, which is known for its simplicity and effectiveness in numerical optimization.

In this study, we developed a novel algorithm named the Secant Optimization Algorithm (SOA), which is based on the idea of approximating the derivative of a function by using secant lines between two points, thereby eliminating the need for gradient information. SOA adapts this principle into an optimization framework, allowing it to efficiently navigate complex objective functions without relying on direct gradient calculations, which are often impractical or costly in real-world applications. By leveraging this derivative-free approach, SOA offers a unique advantage in solving non-convex. In these multi-dimensional optimization problems, traditional methods may struggle with slow convergence or get trapped in local optima.

SOA’s operational mechanism is designed to enhance both the exploration and exploitation phases of optimization, ensuring a well-balanced search process. During exploration, SOA efficiently spans the feasible domain, identifying promising regions, while the exploitation phase allows for a focused search within these regions to find the optimal solution. This balance makes SOA effective in identifying near-optimal solutions without the common pitfalls of stagnation in suboptimal regions.

The reason that the Secant Method has been chosen as the foundation for the Secant Optimization Algorithm (SOA) stems from its specific advantages in optimizing the problems of hyperparameter tuning. Hyperparameter tuning typically encounters non-differentiable objective functions, such as validation errors of machine learning models, where gradient-based methods cannot be applied. The Secant Method, a non-derivative method, approximates derivatives based on function values at two locations, offering an official but pliable way to search high-dimensional, intricate spaces. Compared with typical metaheuristics like Particle Swarm Optimization (PSO) or Genetic Algorithms (GA), which heavily rely on random operators, iterative improvement of the Secant Method provides a principled way of using local knowledge without compromising on computational efficiency. Furthermore, its existence as a population-based metaheuristic introduces a novel mechanism of search that collaborates with existing algorithms, potentially uncovering solutions in regions of the search space that other mechanisms may not approach.

To validate the robustness of the Secant Optimization Algorithm (SOA), it was tested on a diverse array of challenging benchmark functions, including CEC2021 with 20 dimensions and CEC2020 with 50 and 100 dimensions. These benchmarks cover a range of problem types, including low, medium, and high-dimensional and nonlinear problem sets. Additionally, SOA was applied to practical engineering challenges, particularly in the field of power systems, where it successfully addressed parameter identification for photovoltaic (PV) systems.

The results of these tests revealed SOA’s competitive performance when compared with several state-of-the-art optimizers, including both recent and widely cited algorithms known for their robustness and efficiency. SOA demonstrated faster convergence rates and higher accuracy across a range of scenarios, showcasing its adaptability to both single-objective and multi-objective optimization challenges. Furthermore, convergence curves and statistical analyses confirmed SOA’s superior ability to achieve global optima, thus highlighting its potential as a valuable addition to optimization problems.

The main contributions of the paper are listed as follows.

The proposed SOA offers several advantages for global optimization tasks. The first advantage of SOA is its simplicity, as it draws on the iterative, parameter-free nature of the secant method, eliminating the need for complex control parameters in the algorithm's design. The second advantage is its effectiveness in solving a wide range of optimization problems, from high-dimensional to nonlinear challenges, across various scientific fields. A third benefit is SOA’s balanced approach to exploration and exploitation, which enables efficient convergence toward the global optimum, providing robust solutions for decision variables. Finally, SOA demonstrates strong performance in real-world optimization applications, making it a versatile and powerful tool for complex problem-solving tasks.

The remainder of this article is summarized as follows. In Section 2, a thorough literature review is provided. Section 3 provided the basic concepts of the secant method and presented the proposed SOA algorithm while the results and computational complexity are provided in Section 4. Section 5 illustrates the implementation of SOA to solve parameter identification for photovoltaic (PV) system. Section 6 discusses the application of SOA to CNN Hyperparameters Optimization, and several conclusions are drawn in Section 6.

This section provides a brief overview of key population-based metaheuristic algorithms, divided into four main types: evolution-based, swarm intelligence-based, physics/chemistry-based, and mathematics-based approaches. Each category uses different guiding principles to optimize solutions and has been applied in various fields. We focus specifically on mathematics-inspired algorithms.

Evolution-based metaheuristics were among the first metaheuristic algorithms to be created, drawing on natural selection and genetic principles. These algorithms generate a pool of candidate solutions and employ specialized evolutionary operators to generate new solutions through subsequent iterations⁵². One of the most well-known evolution-based metaheuristics is the genetic algorithm (GA)²⁷, which simulates Darwinian natural selection. Differential evolution (DE) ⁵³ is another prominent evolution-based metaheuristic that employs a population of solutions while emphasizing mutation and recombination more strongly. While GA is primarily based on the crossover operator, DE focuses on mutation, making it simpler and, in some situations, more efficient than GA. Aside from GA and DE, several other evolutionary algorithms have been created, including genetic programming (GP) ⁵⁴, evolution strategies (ES) ⁵⁵, gene expression programming (GEP) ⁵⁶, and biogeography-based optimization (BBO) ⁵⁷. Each of these algorithms contains distinct variations on evolutionary principles, extending the breadth of tools accessible for optimization in complicated problem areas ⁵⁸.

Swarm intelligence algorithms are inspired by the collective behaviors observed in nature, including the coordinated actions of animals, birds, insects, and marine organisms⁵⁹. Particle Swarm Optimization (PSO) ⁵⁹, for example, is based on bird flocking behavior, while the Coot optimizer ²⁹ mimics the behavior of coots when they are looking for foods in nature. Chameleon swarm algorithm ⁶⁰ draws from the dynamic behavior of chameleons when navigating and hunting for food sources on trees. Other algorithms include the White Shark Optimizer ⁶¹, inspired by shark predatory tactics, and the snake Optimizer ⁶², which emulates snake movements. The Horse Herd Optimization Algorithm ⁶³ reflects the social behavior of horse herds, and Northern Goshawk Optimization⁶⁴ simulates the behavior of northern goshawk during prey hunting. Additionally, algorithms such as the Slime Mould Algorithm ⁶⁵, Hunger Games Search⁶⁶, Colony Predation Algorithm ⁶⁷, Ant Lion Optimization Algorithm⁶⁸, Flamingo Search Algorithm ⁶⁹, Sparrow Search Algorithm⁷⁰, and sand cat swarm optimization method⁷¹, are inspired by various biological processes, social structures, and survival strategies. These algorithms demonstrate the versatility of swarm intelligence principles, making them highly effective for solving complex optimization problems across diverse fields.

Physical law principles are the foundation for proposing solutions to optimization challenges through physics-based metaheuristic algorithms⁷². Below are some notable and effective optimization algorithms that fall under the physics-based metaheuristic algorithms category. The Lichtenberg Algorithm⁷³, a well-known metaheuristic inspired by the Lichtenberg figures patterns, has been applied across various domains, including data clustering, global optimization, classification tasks, and various engineering designs. The Equilibrium Optimizer (EO) ⁷², which inspired by the physical equation of the mass balance that provides the conservation of mass entering, leaving, and generating in a control volume and the system always reaches an equilibrium point, has garnered significant interest for its applicability in enhancing and resolving a wide range of problems, such as image segmentation, feature selection, and global optimization challenges. Additionally, Plasma Generation Optimization (PGO)⁷⁴ is based on the process of plasma generation. In recent years, PGO has been effectively utilized to address various issues, including global optimization, time series forecasting, and image segmentation. Other physics-based metaheuristic algorithms include, RIME algorithm ³⁰, The Kepler Optimization Algorithm (KOA) ⁷⁵, and etc., each contributing to the field with their unique approaches and applications.

Mathematical-inspired metaheuristic algorithms leverage principles from various mathematical domains to formulate their updating rules and strategies for optimization. These algorithms draw inspiration from fundamental concepts, including arithmetic operations, trigonometric functions, geometric principles, calculus-based algorithms, and others, to create robust optimization techniques capable of solving complex problems. Figure 2 gives the classification of the inspiration of Mathematical-inspired metaheuristic algorithms and examples for each category. The diversity of these mathematical foundations underscores the versatility of mathematical-inspired metaheuristic algorithms, which can be applied across various fields, from engineering and finance to data science and artificial intelligence. By combining insights from arithmetic, trigonometry, and geometry, these algorithms not only improve optimization processes but also pave the way for innovative solutions to real-world challenges.

One of the most widely recognized mathematical algorithms is the population-based Sine Cosine Algorithm (SCA) proposed by Mirjalili in⁷⁶, which utilizes a sinusoidal mathematical framework to generate and distribute candidate solutions that progressively approach the optimal solution. The SCA has been effectively applied to a variety of single-objective nonlinear multivariable benchmark functions, both unimodal and multimodal. Results from these applications have demonstrated SCA's rapid convergence and its ability to avoid local optima. Numerous modifications to the SCA have been proposed and documented in the literature. These adaptations enhance the algorithm's performance and broaden its applicability across various optimization problems. For a comprehensive overview of these advancements, refer to the extensive body of work available in research publications on this topic, which can be found of in ⁷⁷. The SCA and its variants have found extensive applications across various fields, including engineering problems ⁷⁸, Image processing⁷⁹, scheduling ⁸⁰, and different optimization applications ^{81 82 83}. The SCA is valued for its simplicity, efficiency, and ability to solve different complex applications.

The golden sine algorithm (GSA)⁸⁴ is also inspired by trigonometric functions, specifically the sine function, and the mathematical concept of the golden ratio. The GSA is designed to find optimal solutions through an iterative process that balances exploration (broad search across the solution space) and exploitation (focused search around promising regions). By leveraging the golden ratio, a principle that appears frequently in nature, the GSA mimics the efficient structures seen in phenomena like leaf arrangements and shell spirals, helping guide the search process toward high-quality solutions. The GSA is particularly useful for solving complex, nonlinear, and high-dimensional optimization problems. It has been applied in a wide range of fields, including Support vector regression parameter optimization ⁸⁵, Multisensory scheduling for UAV ⁸⁶ Industrial engineering design problems ⁸⁷, and etc.

The Arithmetic Optimization Algorithm (AOA) ⁸⁸uses basic arithmetic operators—addition, subtraction, multiplication, and division—as its core mechanisms. In AOA, multiplication and division are primarily used in the exploration phase due to their tendency to produce widely dispersed values, encouraging a broader search. Conversely, addition and subtraction are applied in the exploitation phase to refine solutions more locally. This structured use of arithmetic operations allows AOA to balance exploration and exploitation effectively for diverse optimization tasks. Due to its straightforward structure, AOA is widely applied in various engineering fields, such as Structural Optimization ⁸⁹, Maximum Power Point Tracking (MPPT) ⁹⁰, Model identification ⁹¹, Image segmentation ⁹², The patient admission scheduling problem (PASP) ⁹³ and other applications can be found in ⁸⁸.

Gradient-Based Optimization (GBO) ⁹⁴ takes inspiration from gradient-based methods like Newton’s method, featuring two main operators: the Gradient Search Rule (GSR) and the Local Escaping Operator (LEO). The GSR helps accelerate convergence by navigating the search space using a gradient-based approach, while the LEO enables the algorithm to escape local optima, enhancing solution quality. Due to its straightforward structure, GBO is widely applied in various engineering fields, scheduling ⁹⁵, Portfolio ⁹⁶, Manage storage space efficiently ⁹⁷ and Adaptive Control⁹⁸,

The Spherical Evolution (SE) algorithm ⁹⁹ is a novel, math-based optimization technique that employs a spherical search pattern, guiding the movement of individuals across a wider, more diverse search space than traditional methods. This extended search capability enhances its performance, making SE particularly noteworthy in optimization research. SE has been applied to various real-world problems, demonstrating promising results in areas such as energy optimization, layout design, and parameter estimation. Its flexibility and efficiency in navigating complex solution spaces are central to its growing use in optimization tasks. For example, Wind Farm Layout Optimization ¹⁰⁰, Solar Cell and Photovoltaic Model Optimization ^{101 102},and etc.

The Circle Optimization Algorithm (COA) ¹⁰³ is a math-based metaheuristic inspired by the geometry and symmetry of a circle, designed to tackle continuous optimization problems. It models search agents as points distributed on a circle, moving iteratively along circular paths to balance exploration of the solution space and exploitation of promising areas. COA has demonstrated effective performance in various optimization applications and can be combined with other metaheuristics to enhance results. For example, Maximum power point tracking (MPPT) ¹⁰⁴, Kernel extreme learning machine optimization ¹⁰⁵, Proton exchange membrane fuel cell modeling ¹⁰⁶ and etc.

Runge-Kutta Optimization Algorithm (RUN) ¹⁰⁷ is one of the most cited math-based algorithm, inspired by the RK4 method of slope calculation from differential equations, where these slopes guide the search for optimal solutions. An enhanced solution quality mechanism helps avoid local optima and speeds up convergence, making RUN particularly effective for complex, nonlinear, and high-dimensional problems. This approach is especially valuable in applications like renewable energy system design ¹⁰⁸, control system optimization ¹⁰⁹, and machine learning ¹¹⁰. There are a lot of other applications that use the RUN algorithm due to the algorithm’s adaptability and effectiveness.

Exponential Distribution Optimizer (EDO) ³⁴ is also a novel math-inspired algorithm designed for global optimization and engineering problems. It draws its principles from the exponential distribution, which models the time between events in a Poisson process. In the context of optimization, EDO uses exponential distribution to guide the search process by balancing exploration and exploitation. EDO has been shown to be effective in a variety of engineering applications, such as parameter estimation of proton exchange membrane fuel cells (PEMFCs) ¹¹¹, power systems optimization ¹¹², multi-level image segmentation ¹¹³ and etc., where precision, robustness, and computational efficiency are crucial. The algorithm’s adaptability and effectiveness make it a promising tool for solving real-world problems in diverse fields.

Newton- Raphson -Based Optimizer (NRBO) ¹¹is inspired by the Newton- Raphson method and explores the search space using two key strategies: the Newton- Raphson search Rule and the Trap Avoid Operator, along with several matrix- based groups to enhance solution refinement. NRBO has shown effectiveness in cross a variety of real-world applications, including image segmentation, parameter estimation, wireless communication networks, and engineering control systems (EC) .

Triangulation Topology Aggregation Optimizer (TTAO)³² is rooted in the mathematical concept of similar triangular topology. TTAO integrates two main strategies—generic aggregation and local aggregation—which collaboratively enable the iterative construction of multiple similar triangular topological structures. These strategies are designed to achieve a robust balance between exploration and exploitation throughout the optimization process. TTOA has also shown effectiveness in cross a variety of real-world applications.

The advancements in math-inspired algorithms underscore the innovative integration of mathematical principles into optimization techniques. From classical methods like the Runge-Kutta and quadratic interpolation to more recent developments like the Exponential Distribution Optimizer (EDO) and the Golden Sine Algorithm (GSA), each approach leverages unique mathematical properties to improve convergence, robustness, and adaptability in complex solution spaces. These algorithms provide powerful tools for solving diverse real-world problems by embedding mathematical strategies into their search mechanisms.

In the field of hyperparameter tuning, metaheuristic algorithms like Particle Swarm Optimization (PSO), Genetic Algorithms (GA), and Differential Evolution (DE) have been widely used since they can comfortably deal with multimodal and non-differentiable objective functions. These do have the drawback of requiring control parameter tuning (like inertia weight for PSO or GA's crossover rate) and can get computationally intensive for high-dimensional problems. Contrarily, the Secant Method offers a derivative-free, sparsely parameterized approach with local function values to estimate derivatives and an intrinsic systematic search process. Transposition of the method as a population-based metaheuristic allows the proposed Secant Optimization Algorithm (SOA) to combine the efficiency of the Secant Method with metaheuristics' global search capability and is therefore particularly suited to hyperparameter optimization problems where computational effort and solution accuracy are crucial. Building on this foundation, the proposed algorithm, inspired by the secant method, introduces a novel approach, aiming to harness the iterative efficiency of the secant method to tackle optimization challenges with enhanced precision and reduced computational complexity.

In addition, the recent advancements in metaheuristics have focused on improving exploration, exploitation, and diversity maintenance. For instance, the paper of ¹¹⁴ introduces surrogate-assisted differential evolution with region decomposition and gradient-based mutation to effectively locate multiple optima in expensive problems. Similarly, the paper ¹¹⁵enhances constraint satisfaction and optimization by using gradient-based repair techniques.

Additionally, techniques such as in the work of ¹¹⁶and ¹¹⁷focus on enhancing population diversity, which is critical for efficiently navigating constrained and multimodal optimization landscapes. These approaches, which balance exploration, exploitation, and diversity, inspire our proposed algorithm, highlighting future research directions for improving metaheuristic performance in complex problem spaces. This addition contributes to the growing field of math-based optimizers, further expanding the toolkit for addressing high-dimensional, nonlinear, and multimodal optimization problems across various applications.

The Secant Method was the motivation behind the Secant Optimization Algorithm (SOA) as it operates effectively to estimate derivatives without gradient information—a significant point of strength in hyperparameter optimization, where the goal functions may come with noisy or non-differentiable settings. While SOA's stochastic search strategy guarantees aggressive global search capability, the Secant Method's iterative optimization from secant lines between two points offers a computationally effective way of leveraging local knowledge. With the synergy between the two, SOA is well-suited to navigate the high-dimensional, intricate search spaces encountered during hyperparameter tuning and presents an alternative viewpoint to more conventional metaheuristics like PSO or GA. The Secant method is an efficient numerical technique for solving nonlinear equations within the domain of numerical analysis. It is recognized as one of the local search methods and ranks among the key optimization algorithms used to find optimal solutions, alongside Newton's method, quasi-Newton methods, and others in this field ¹¹⁸. Unlike Newton's method, the Secant method relies on the secant property and utilizes two initial approximations along with the function’s first derivative, instead of requiring the second derivative. This approach iteratively approximates the function’s first derivative to zero by leveraging consecutive points, making it a relatively effective technique for numerical optimization¹¹⁹.

If we have is a quadratic function and we approximate it by the Taylor series at :

Taking the derivative of Eq. (1) and since we want to find the minimum point of the function , so we setting the result equal to zero :

Equation (2) can be expressed as follows:

Where Eq. (3) represents an approximation of the function as a linear function between the two approximation points and of the minimum point , with represent the slope of line that joins the points and .

As a result, Eq. (3) can be interpreted as offering a new approximation for the roots of as follows:

So, the Eq. (5) can be expressed using the points and to be:

Thus, Eq. (6) is the general formula for the Secant Method, This formulation highlights the iterative nature of the Secant Method, where each new approximation is derived from the previous two approximations and by iterating this process, the method refines its estimates and ideally converges to the actual root of the equation . The process of selecting the next approximation in the Secant Method can be illustrated in Fig. 3.

The strength of the Secant Method lies in its ability to leverage only function values, making it a preferred choice in scenarios where derivatives are not readily available or computationally expensive to obtain. By using successive points to improve the approximation, it can achieve faster convergence than simpler methods like the bisection method.

The Secant Optimization Algorithm is based on the Secant Method, a mathematical technique that employs iterative refining to find optimal solutions. The algorithm combines two basic processes: exploration and exploitation, which ensures a balance between wide searching and solution refinement. During the exploration phase, the algorithm leverages the systematic update mechanism of the Secant Method to guide the search toward promising areas. By utilizing local problem-specific information, it precisely exploits current solutions and progressively improves them through incremental updates. In the exploitation phase, the algorithm introduces diversity by incorporating random movements and searching in unexplored regions of the solution space. This helps prevent premature convergence to local optima. This dual-process design enhances the algorithm’s ability to achieve global optimization effectively.

In contrast, traditional metaheuristic algorithms often combine random exploration with local searches, typically confined to small neighborhoods within the solution space. While these strategies can be effective in certain contexts, they may struggle to escape local optima due to their inherent dependence on randomness. Recognizing these limitations, we propose a novel algorithm, the Secant-Based Optimizer.

This innovative approach allows for unrestricted and stochastic movement within the solution space, enabling the exploration of a wider range of potential solutions without directly relying on derivatives. By employing derivative approximations instead of actual derivatives, the Secant-Based Optimizer introduces enhanced diversity and flexibility into the optimization process. This capability not only improves the algorithm's efficiency in traversing complex landscapes but also accelerates convergence toward the desired objective.

Integrating the foundational principles of the Secant Method with a novel exploration strategy, the SOA is designed to expand the search for solutions. This synergy enhances performance and adaptability, making the algorithm a versatile tool for addressing complex optimization challenges across various fields, including engineering, finance, and computational science.

The SOA is a population-based optimization algorithm that leverages the principles of the Secant Method to explore the solution space. In this proposed algorithm, the population consists of a set of vectors that represent potential solutions to the optimization problem. The SOA algorithm iteratively refines these solutions over successive generations. The following three key operations are employed to update the positions of the vectors in each generation:

In this framework, the objective is to minimize the target function, illustrating the effectiveness of the SOA in navigating complex optimization landscapes.

This section details simulation studies conducted to assess the Secant Optimization Algorithm (SOA) in terms of optimization effectiveness. SOA was evaluated rigorously across two distinct CEC benchmark suites, each designed with unique dimensional challenges. The initial evaluation used the CEC2021 benchmark, featuring 20-dimensional test functions. This lower-dimensional setup allowed an initial assessment of SOA’s efficiency in simpler search spaces. To further explore its robustness, SOA was tested on the more challenging CEC2020 suite, which includes high-dimensional test functions. The algorithm was evaluated at 50 dimensions to simulate intermediate complexity and at 100 dimensions for high-complexity testing, examining SOA’s scalability and performance as the dimensionality and complexity of the problem increased. Specific details on the CEC2021 and CEC2020 benchmarks, as well as the experimental setup, are available in references ¹²⁰ and, ¹²¹respectively.

To ensure a comprehensive performance evaluation, SOA was benchmarked against two distinct groups of algorithms. The first group consisted of 11 mathematically inspired metaheuristic algorithms, while the second included 9 optimizers, encompassing both newly developed algorithms and variants of existing techniques. This broad comparison provides insights into SOA’s competitive standing within the latest advancements in metaheuristics. These evaluations across multiple dimensions offer an extensive analysis of SOA’s strengths and areas for further improvement across various optimization complexities.

The selection of comparative algorithms was guided by their diversity in design principles and relevance to recent developments in the metaheuristics field. The first group of recent and variant metaheuristics includes algorithms like FATA, ArchOA, and HHO, which represent modern adaptations and hybridizations of established optimization paradigms. QleSCA integrates reinforcement learning principles into the Sine Cosine Algorithm, demonstrating innovation in combining learning techniques with classical metaheuristics. These algorithms were chosen to assess SOA’s competitiveness against both novel methodologies and improved variants of traditional approaches.

The second group comprises mathematically inspired metaheuristics, such as the Arithmetic Optimization Algorithm (AOA), Chaos Game Optimization (CGO), and Circle Search Algorithm (CircleSA), which emphasize mathematical models as the foundation of their exploration and exploitation mechanisms. These algorithms, including CDO and EVO, were selected for their diverse inspiration sources and problem-solving strategies. For example, CGO applies chaos theory to enhance exploration, while EO uses equilibrium-based strategies for balancing search phases. This diversity ensures a robust comparative framework, reflecting SOA’s performance against a wide spectrum of optimization techniques. By benchmarking SOA against such a varied set of algorithms, the evaluation not only highlights SOA’s advantages in adaptability and robustness but also identifies specific challenges it may face in different optimization landscapes. This justifies the choice of these algorithms as representative benchmarks in numerical experiments and performance analysis.

Parameter settings were applied for the comparative algorithms as recommended in their original studies, with specific configurations summarized in Tables 1 and 2. During testing, all algorithms used a fixed population size of 30, with a maximum of 2500 evaluations. Experiments were conducted in Python on a high-performance Rocky Linux 9.4 (Blue Onyx) system. The computational setup included a machine with 1 TiB of RAM and an AMD EPYC 7713 64-core Processor, providing a stable and efficient environment for consistent testing.

Each algorithm, being stochastic, was independently run 30 times to mitigate variability in results. Performance evaluation was based on the fitness values from these runs, using the average (Avg), standard deviation (std), median (Med), Friedman ranking, and the Wilcoxon signed-rank test to establish a comprehensive ranking for each algorithm. This approach ensures a robust comparison by considering both central tendency and variability in performance.

In order to show the real-world usability of the Secant Optimization Algorithm (SOA), we used it to optimize the hyperparameters of a convolutional neural network (CNN) for image classification problems. This meets the criterion of realistic problem instances, since hyperparameter optimization is a classic problem in machine learning with real-world applications in computer vision and other areas. We evaluated SOA-CNN on four datasets: MNIST, MNIST-RD, Convex, and Rectangle-I, real-world-inspired image classification tasks of different complexities [30]. Table 23 lists the specifications of the datasets, such as input size, number of classes, and train/test splits.

The tests were conducted on a powerful Rocky Linux 9.4 (Blue Onyx) system with 1 TiB RAM and an AMD EPYC 7713 64-core Processor for stable and effective testing. All the algorithms, SOA and the 19 state-of-the-art comparison algorithms enumerated in Section 4.3, were set with a population size of 30 and 2500 as the maximum number of function evaluations, whereas SOA was configured for 10 iterations. The hyperparameters of the CNN architecture optimized by SOA include convolutional layer output channels, dropout rate, fully connected layer units, and learning rate.

Table 24 lists the ranges of these hyperparameters, which significantly impact CNN performance by managing model capacity, regularization, and training dynamics.The influence of these hyperparameters is as follows: increased convolutional output channels (conv1_out_channels, conv2_out_channels, conv3_out_channels) enhance feature extraction at the cost of computation; an increased dropout_rate prevents overfitting; fc1_units regulate the capacity of the model in the dense layer; and the learning_rate regulates the rate of convergence. The suitable adjustment of these parameters, achieved by SOA, balances model accuracy and generalization.

Table 25 presents the quantitative SOA-CNN results, including accuracy, precision, recall, F1-score (macro-averaged), and the optimal hyperparameters for each dataset. The optimal hyperparameter order is as follow: [conv1_out_channels, conv2_out_channels, conv3_out_channels, dropout_rate, fc1_units, learning_rate]. SOA-CNN performs excellently well, achieving 99.40% on MNIST, 96.24% on MNIST-RD, 97.39% on Convex, and 98.27% on Rectangle-I, demonstrating its capacity to handle complex hyperparameter tunings.

The comparison with advanced competitive methods, shown in Table 26, highlights SOA-CNN’s competitive edge. For instance, SOA-CNN outperforms most methods on MNIST-RD (96.24%) and Rectangle-I (98.27%), while remaining comparable to CPOCNN on MNIST (99.40% vs. 99.77%). These results underscore SOA’s ability to deliver robust solutions for realistic machine learning tasks, complementing its performance on synthetic benchmarks and PV system optimization.

Secant Optimization Algorithm (SOA) is a revolutionary new development in math-inspired metaheuristic optimization that skillfully blends historical mathematical theory and modern computational style. By transferring the derivative-free iterative efficacy of the Secant Method into the population-based framework, SOA achieves an optimal balance of exploration and exploitation, minimizing its reliance on randomness while maximizing variability in complex high-dimensional solution spaces. Comparative experiments on the CEC2021 and CEC2020 benchmark suites, as well as on benchmarks against 11 mathematically inspired metaheuristics and 9 state-of-the-art optimizers, demonstrate SOA's scalability, stability, and competitive performance across a broad spectrum of problem dimensionalities. Moreover, its successful implementation to optimize Single-Diode, Double-Diode, and full PV Module models demonstrates its practical applicability in solving real-world problems, particularly in energy systems. These findings verify the usefulness of SOA as a versatile, powerful tool with broad applicability across fields such as engineering, finance, and computational science. Through its provision of a computationally sound extension to traditional metaheuristics like PSO and GA, particularly for hyperparameter optimization and multimodal, non-differentiable problems, SOA contributes meaningfully to the evolving state of optimization methods. Future research can explore further enhancements to SOA's framework and its extension to other real-world issues, solidifying its place in further advancing optimization methods.