Skip to content

Commit e1d2910

Browse files
Revise theory content for exponential function optimization
Updated the title and improved the clarity of the text throughout the document, including mathematical explanations and descriptions of techniques for optimizing exponential function computations.
1 parent 4d6778b commit e1d2910

1 file changed

Lines changed: 18 additions & 11 deletions

File tree

  • content/learning-paths/servers-and-cloud-computing/fexpa
Lines changed: 18 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,16 @@
11
---
2-
title: Theory
2+
title: Learn exponential function optimization techniques
33
weight: 2
44

55
### FIXED, DO NOT MODIFY
66
layout: learningpathall
77
---
88

99
## The exponential function
10-
The exponential function is a fundamental mathematical function used across a wide range of algorithms for signal processing, High-Performance Computing and Machine Learning. Optimizing its computation has been the subject of extensive research for decades. The precision of the computation depends both on the selected approximation method and on the inherent rounding errors associated with finite-precision arithmetic, and it is directly traded off against performance when implementing the exponential function.
10+
The exponential function is a fundamental mathematical function used across a wide range of algorithms for signal processing, High-Performance Computing and Machine Learning. Researchers have extensively studied optimizing its computation for decades. The precision of the computation depends both on the selected approximation method and on the inherent rounding errors associated with finite-precision arithmetic, and it is directly traded off against performance when implementing the exponential function.
1111

1212
## Range reduction
13-
Polynomial approximations are among the most widely used methods for software implementations of the exponential function. The accuracy of a Taylor series approximation for exponential function can be improved with the polynomials degree but will always deteriorate as the evaluation point moves further from the expansion point. By applying range reduction techniques, the approximation of the exponential function can however be restricted to a very narrow interval where the function is well-conditioned. This approach consists in reformulating the exponential function in the following way:
13+
Polynomial approximations are among the most widely used methods for software implementations of the exponential function. The accuracy of a Taylor series approximation for exponential function can be improved with the polynomial's degree but deteriorates as the evaluation point moves further from the expansion point. By applying range reduction techniques, you can restrict the approximation of the exponential function to a very narrow interval where the function is well-conditioned. This approach reformulates the exponential function in the following way:
1414

1515
$$e^x=e^{k×ln2+r}=2^k \times e^r$$
1616

@@ -22,16 +22,16 @@ Since k is an integer, the evaluation of 2^k can be efficiently performed using
2222

2323
$$e^x \approx 2^k \times p(r)$$
2424

25-
It is important to note that the polynomial p(r) is evaluated exclusively over the interval [-ln2/2, +ln2/2]. So, the computational complexity can be optimized by selecting the polynomial degree based on the required precision of p(r) within this narrow range. Rather than relying on a Taylor polynomial, a minimax polynomial approximation can be used to minimize the maximum approximation error over the considered interval.
25+
The polynomial p(r) is evaluated exclusively over the interval [-ln2/2, +ln2/2]. So, the computational complexity can be optimized by selecting the polynomial degree based on the required precision of p(r) within this narrow range. Rather than relying on a Taylor polynomial, a minimax polynomial approximation can be used to minimize the maximum approximation error over the considered interval.
2626

2727
## Decomposition of the input
28-
The decomposition of an input value as x = k × ln2 + r can be done in 2 steps:
28+
Decompose an input value as x = k × ln2 + r in two steps:
2929
- Compute k as: k = round(x⁄ln2), where round(.) is the round-to-nearest function
3030
- Compute r as: r = x - k × ln2
3131

32-
Rounding of k is performed by adding an adequately chosen large number to a floating-point value and subtracting it just afterward (the original value is rounded due to the finite precision of floating-point representation). Although explicit rounding instructions are available in both SVE and SME, this method remains advantageous as the addition of the constant can be fused with the multiplication by the reciprocal of ln2. This approach assumes however that the floating-point rounding mode is set to round-to-nearest, which is the default mode in Armv9-A. By integrating the bias into the constant, 2^k can also be directly computed by shifting the intermediate value.
32+
Rounding of k is performed by adding an adequately chosen large number to a floating-point value and subtracting it just afterward (the original value is rounded because of the finite precision of floating-point representation). Although explicit rounding instructions are available in both SVE and SME, this method remains advantageous because the addition of the constant can be fused with the multiplication by the reciprocal of ln2. This approach assumes that the floating-point rounding mode is set to round-to-nearest, which is the default mode in Armv9-A. By integrating the bias into the constant, 2^k can also be directly computed by shifting the intermediate value.
3333

34-
Rounding error during the second step will introduce a global error as we will have:
34+
A rounding error during the second step introduces a global error as you have:
3535

3636
$$ x \approx k \times ln2 + r $$
3737

@@ -44,20 +44,27 @@ $$ (-1)^s \times 2^{(exponent - bias)} \times (1.fraction)_2 $$
4444

4545
where s is the sign bit and 1.fraction represents the significand.
4646

47-
The value 2^k can be encoded by setting both the sign and fraction bits to zero and assigning the exponent field the value k + bias. If k is an 8-bits integer, 2^k can be efficiently computed by adding the bias value and positioning the result into the exponent bits of a 32-bit floating-point number using a logical shift.
47+
The value 2^k can be encoded by setting both the sign and fraction bits to zero and assigning the exponent field the value k + bias. If k is an 8-bit integer, 2^k can be efficiently computed by adding the bias value and positioning the result into the exponent bits of a 32-bit floating-point number using a logical shift.
4848

49-
Taking this approach a step further, a fast approximation of exponential function can be achieved using bits manipulation techniques alone. Specifically, adding a bias to an integer k and shifting the result into the exponent field can be accomplished by computing an integer i as follows:
49+
Taking this approach a step further, you can achieve a fast approximation of exponential function using bit manipulation techniques alone. Specifically, adding a bias to an integer k and shifting the result into the exponent field can be accomplished by computing an integer i as follows:
5050

5151
$$i=2^{23} \times (k+bias) = 2^{23} \times k+2^{23} \times bias$$
5252

5353
This formulation assumes a 23-bit significand, but the method can be generalized to other floating-point precisions.
5454

55-
Now, consider the case where k is a real number. The fractional part of k will propagate into the significand bits of the resulting 2^k approximation. However, this side effect is not detrimental, it effectively acts as a form of linear interpolation, thereby improving the overall accuracy of the approximation. To approximate the exponential function, the following identity can be used:
55+
Now, consider the case where k is a real number. The fractional part of k propagates into the significand bits of the resulting 2^k approximation. However, this side effect isn't detrimental; it effectively acts as a form of linear interpolation, thereby improving the overall accuracy of the approximation. To approximate the exponential function, use the following identity:
5656

5757
$$e^x = 2^{x⁄ln2}$$
5858

5959
As previously discussed, this value can be approximated by computing a 32-bit integer:
6060

6161
$$i = 2^{23} \times x⁄ln2 + 2^{23} \times bias = a \times x + b $$
6262

63-
Continue to the next section to make a C-based implementation of the exponential function.
63+
## What you've accomplished and what's next
64+
65+
In this section, you learned the mathematical foundations for optimizing exponential functions:
66+
- Range reduction techniques that narrow the evaluation interval
67+
- How to decompose inputs using k × ln2 + r reformulation
68+
- Bit manipulation techniques for computing scaling factors
69+
70+
Next, you'll implement these concepts in C using SVE intrinsics.

0 commit comments

Comments
 (0)