Skip to content

Commit d97a798

Browse files
committed
Use consistent Markdown style
1 parent bc84c2a commit d97a798

1 file changed

Lines changed: 33 additions & 22 deletions

File tree

README.md

Lines changed: 33 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,10 @@
33
Most strings online are in unicode using the UTF-8 encoding. Validating strings
44
quickly before accepting them is important.
55

6-
7-
86
## How to use is_utf8
97

10-
This is a simple one-source file library to validate UTF-8 strings at high speeds using SIMD instructions. It works on all platforms (ARM, x64).
8+
This is a simple one-source file library to validate UTF-8 strings at high
9+
speeds using SIMD instructions. It works on all platforms (ARM, x64).
1110

1211
Build and link `is_utf8.cpp` with your project. Code usage:
1312

@@ -20,30 +19,43 @@ Build and link `is_utf8.cpp` with your project. Code usage:
2019

2120
It should be able to validate strings using less than 1 cycle per input byte.
2221

23-
Real-word usage
24-
------
25-
26-
This C++ library is part of the JavaScript package [utf-8-validate](https://github.com/websockets/utf-8-validate). The utf-8-validate package is routinely downloaded more than [a million times per week](https://www.npmjs.com/package/utf-8-validate).
22+
## Real-word usage
2723

28-
If you are using Node JS (19.4.0 or better), you already have access to this function as `buffer.isUtf8(...)`.
24+
This C++ library is part of the JavaScript package
25+
[utf-8-validate](https://github.com/websockets/utf-8-validate). The
26+
utf-8-validate package is routinely downloaded more than
27+
[a million times per week](https://www.npmjs.com/package/utf-8-validate).
2928

29+
If you are using Node JS (19.4.0 or better), you already have access to this
30+
function as `buffer.isUtf8(...)`.
3031

32+
## Requirements
3133

32-
Requirements
33-
-------
34-
35-
- C++11 compatible compiler. We support LLVM clang, GCC, Visual Studio. (Our optional benchmark tool requires C++17.)
34+
- C++11 compatible compiler. We support LLVM clang, GCC, Visual Studio. (Our
35+
optional benchmark tool requires C++17.)
3636
- For high speed, you should have a recent 64-bit system (e.g., ARM or x64).
3737
- If you rely on CMake, you should use a recent CMake (at least 3.15).
38-
- AVX-512 support require a processor with AVX512-VBMI2 (Ice Lake or better) and a recent compiler (GCC 8 or better, Visual Studio 2019 or better, LLVM clang 6 or better). You need a correspondingly recent assembler such as gas (2.30+) or nasm (2.14+): recent compilers usually come with recent assemblers. If you mix a recent compiler with an incompatible/old assembler (e.g., when using a recent compiler with an old Linux distribution), you may get errors at build time because the compiler produces instructions that the assembler does not recognize: you should update your assembler to match your compiler (e.g., upgrade binutils to version 2.30 or better under Linux) or use an older compiler matching the capabilities of your assembler.
38+
- AVX-512 support require a processor with AVX512-VBMI2 (Ice Lake or better) and
39+
a recent compiler (GCC 8 or better, Visual Studio 2019 or better, LLVM clang 6
40+
or better). You need a correspondingly recent assembler such as gas (2.30+) or
41+
nasm (2.14+): recent compilers usually come with recent assemblers. If you mix
42+
a recent compiler with an incompatible/old assembler (e.g., when using a
43+
recent compiler with an old Linux distribution), you may get errors at build
44+
time because the compiler produces instructions that the assembler does not
45+
recognize: you should update your assembler to match your compiler (e.g.,
46+
upgrade binutils to version 2.30 or better under Linux) or use an older
47+
compiler matching the capabilities of your assembler.
3948

4049
## Reference
4150

42-
- John Keiser, Daniel Lemire, [Validating UTF-8 In Less Than One Instruction Per Byte](https://arxiv.org/abs/2010.03090), Software: Practice & Experience 51 (5), 2021
51+
- John Keiser, Daniel Lemire,
52+
[Validating UTF-8 In Less Than One Instruction Per Byte](https://arxiv.org/abs/2010.03090),
53+
Software: Practice & Experience 51 (5), 2021
4354

44-
### Want more?
55+
## Want more?
4556

46-
If you want a wide range of fast Unicode function for production use, you can rely on the simdutf library. It is as simple as the following:
57+
If you want a wide range of fast Unicode function for production use, you can
58+
rely on the simdutf library. It is as simple as the following:
4759

4860
```C++
4961
#include "simdutf.cpp"
@@ -64,12 +76,11 @@ int main(int argc, char *argv[]) {
6476
6577
See https://github.com/simdutf/
6678
67-
6879
## License
6980
70-
This library is distributed under the terms of any of the following
71-
licenses, at your option:
81+
This library is distributed under the terms of any of the following licenses, at
82+
your option:
7283
73-
* Apache License (Version 2.0) [LICENSE-APACHE](LICENSE-APACHE),
74-
* Boost Software License [LICENSE-BOOST](LICENSE-BOOST), or
75-
* MIT License [LICENSE-MIT](LICENSE-MIT).
84+
- Apache License (Version 2.0) [LICENSE-APACHE](LICENSE-APACHE),
85+
- Boost Software License [LICENSE-BOOST](LICENSE-BOOST), or
86+
- MIT License [LICENSE-MIT](LICENSE-MIT).

0 commit comments

Comments
 (0)