Skip to content
This repository was archived by the owner on May 11, 2026. It is now read-only.

Commit 38aeceb

Browse files
brod-intelbrod-intel
committed
v2.5.0
1 parent d5e88d7 commit 38aeceb

52 files changed

Lines changed: 1545 additions & 189 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.bdsignore.all‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
artifacts
2+
avvdat.ini
3+
vsreports

‎.gitignore‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,3 +44,10 @@ dockerfiles/uos/lib/docker
4444

4545
# Configuration that should never be committed to git
4646
# conf/config.yml
47+
conf/secrets.yml
48+
49+
#used for core
50+
dockerfiles/core/files/
51+
52+
#used for certbot
53+
dockerfiles/certbot/scripts/

‎CHANGELOG.md‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# Change Log
2+
All notable changes to this project will be documented in this file.
3+
4+
The format is based on [Keep a Changelog](http://keepachangelog.com/)
5+
and this project adheres to [Semantic Versioning](http://semver.org/).
6+
7+
## [Unreleased][1.0.0] - 2019-04-24
8+
### Initial internal publication
9+
- An easy way to provision target systems that are bare metal or virtual machines using a just-in-time provisioning process over PXE
10+
11+
## [1.2.0] - 2020-04-01
12+
### Added
13+
- Created a proper README.md file
14+
- Samba service is now availble to allow Windows Profiles to mount ESP directory
15+
- Introduced Edge Software Provisioner Utility Operating System called ESP UOS
16+
- Now automatically detects network settings if omitted in conf/config.yml
17+
18+
### Changed
19+
- Updated error handling
20+
21+
## [1.3.0] - 2020-11-04
22+
### Added
23+
- Bash yaml parsing
24+
- ISO PXE Booting
25+
26+
### Changed
27+
- Fixed Samba mmounting
28+
- Set the Linuxkit version to v0.8
29+
30+
## [1.5.0] - 2021-01-26
31+
### Added
32+
- Rebranded product name
33+
- In-line caching of RPMs, DEBs, TAR and other package management applications
34+
- You can now pre-build certain tasks, for example compile a kernel or mirror a repo on ESP
35+
- Mirroring GitHub repos on ESP
36+
- Virtual PXE - can test profiles or create VMs directly in ESP. Can be used in a Jenkins pipeline for testing ESP Profiles
37+
- Added multiple kernel support
38+
- Input validation types for profile configuration
39+
40+
### Changed
41+
- Improved Nginx and web services
42+
- Improved UOS
43+
- Kernel version
44+
- Error handling to console
45+
- README.md
46+
- Improved UOS process
47+
- Fixed ISO mounting and unmounting
48+
- Improved support for different system BIOS and uEFI
49+
50+
## [1.5.1] - 2021-02-09
51+
### Added
52+
- Let's Encrypt to generate public certificates
53+
- Introduced TLS for all ESP services
54+
55+
### Changed
56+
- Fixed miscellaneous bugs
57+
58+
## [1.6.0] - 2021-02-09
59+
### Added
60+
- Ability to change kernel from different Linux distros, defaults to Clear Linux
61+
- Proxy support to docker-compose.yml
62+
- Github mirror to docker-compose.yml
63+
- Podman support for Red Hat
64+
65+
### Changed
66+
- Gitea is built during the build.sh process instead of run.sh
67+
- Default config.yml to latest LTS Ubuntu
68+
- Fixed miscellaneous bugs
69+
70+
## [1.6.1] - 2021-05-27
71+
### Changed
72+
- UOS now ignores self-signed certificate on ESP
73+
- Gitea startup processes
74+
75+
## [1.6.2] - 2021-06-30
76+
### Changed
77+
- UOS Display name
78+
- login support to UOS when there is an error
79+
80+
## [2.0.0] - 2021-08-27
81+
### Added
82+
- Create Bootable USB to provision devices with no PXE support or ethernet. See “Bootable USB” in the README.md.
83+
- Flash USB Utility to protect from overwriting the wrong drive.
84+
- Utility OS has been rebranded to Micro OS – uOS.
85+
- uOS now supports WiFI and Mobile Cell phone network deployments.
86+
- TLS encryption enabled using self-signed certificate including optional Let’s Encrypt for Web Services.
87+
- All other services except PXE Boot are TLS enabled.
88+
- New ESP one line start command; instead of having to build ESP container images every install, you can start ESP from a single command line. See step 8 of “Quick Installation Guide” in the README.md.
89+
- ESP Core service now monitors for config.yml file changes and automatically runs build.sh command when a change occurs
90+
- ESP supports the ability to provision target devices while being disconnected from the internet.
91+
- Ubuntu Profile now supports config.yml variables network=[default|bridged|network-manager], wifissid= and wifipsk=
92+
- Ubuntu Profile will now search for a Debian mirror on ESP to pull packages directly.
93+
- Virtual PXE now supports building VMs inside a container for distribution and execution on Docker. See ./vpxe.sh -h
94+
95+
### Changed
96+
- uOS Kernel Selection – you can now choose different kernels for the uOS kernel. See ./build.sh -h
97+
- Updated Podman to support network proxies
98+
- Fixed miscellaneous bugs
99+
100+
## [2.0.1] - 2021-10-08
101+
### Added
102+
- "--skip-memory" to makeusb.sh to skip memory check for systems with small off memory.
103+
104+
### Changed
105+
- Fixed typo in the help
106+
- Fixed build ESP containers behind proxy
107+
- Upgraded container base version of nginx, gitea, core and certbot to address CVEs
108+
- Fixed Docker-in-Docker /dev/null deletion on build failing behind proxy
109+
- Fixed detection of failure of docker-builder program to restart
110+
- Fixed makeusb.sh creating legacy BIOS USB images not booting correctly in QEMU
111+
- Fixed squid caching of Linux distro packages
112+
- Fixed miscellaneous bugs
113+
114+
## [2.0.2] - 2021-10-12
115+
### Changed
116+
- Fixed Certbot cert renewal detection
117+
- Fixed Gitea database initialization
118+
- Fixed Podman run for Gitea
119+
- Fixed Podman run for Certbot
120+
121+
## [2.0.3] - 2021-11-19
122+
### Added
123+
- Environment variable for NO_PROXY during build.sh
124+
- Ability to specify Git TAG Names for branches in config.yml
125+
126+
### Changed
127+
- Fixed /dev/null being deleted
128+
- Fixed CVE is in Dockerfiles
129+
- Fixed missing DOCKER_RUN_ARGS
130+
- Fixed Miscellaneous typos
131+
132+
## [2.5.0] - 2022-06-17
133+
### Added
134+
- Dynamic Profiles - The Dynamic Profile feature allows ESP to install software on a target machine without any user interaction. See https://github.com/intel/Edge-Software-Provisioner#dynamic-profile
135+
- Build Red Hat kernels into ESP uOS using Podman. See ./build.sh -k
136+
- Now can designate the interface ESP to listen on for all DHCP requests. See https://github.com/intel/Edge-Software-Provisioner/blob/master/conf/config.yml
137+
- Signed Kernels and Secure Boot ESP uOS will be released in the next version
138+
- Can dynamically inject secretes using environment variables. See https://github.com/intel/Edge-Software-Provisioner/blob/master/conf/secrets.sample.yml
139+
- Additional support for air-gapped environments
140+
- The ability to resume an ESP Profile deployment after failure instead of starting all over. To enable, add kernel parameter "resume=true" in the profile config.yml
141+
- Can specify an ethernet interface for ESP to listen on when running on system with more than one ethernet interface. See https://github.com/intel/Edge-Software-Provisioner/blob/master/conf/config.yml
142+
143+
### Changed
144+
- Updated all kernels to 5.17 and introduced an Intel kernel for latest hardware
145+
- Fixed missing /dev/null when Docker cleans up mounts
146+
- Proxy problems fixed - missing "no_proxy" values were not being passed to all containers
147+
- Ensure previously mounted ISO images are properly unmounted
148+
- Fixed Nginx bugs that stopping bootstrapping in different situations
149+
- Enhanced Code Quality
150+
151+
### Known Issue
152+
- Virtual PXE (vpxe.sh) may cause a kernel panic under a nested VM. Work around is to build a different kernel. For example, `./build.sh -k ubuntu -P`
153+
154+
155+
156+
157+
[1.5.1]: https://github.com/intel/Edge-Software-Provisioner/compare/v1.5...v1.5.1
158+
[1.6.0]: https://github.com/intel/Edge-Software-Provisioner/compare/v1.5.1...v1.6
159+
[1.6.1]: https://github.com/intel/Edge-Software-Provisioner/compare/v1.6...v1.6.1
160+
[1.6.2]: https://github.com/intel/Edge-Software-Provisioner/compare/v1.6.1...v1.6.2
161+
[2.0.0]: https://github.com/intel/Edge-Software-Provisioner/compare/v1.6.2...v2.0
162+
[2.0.1]: https://github.com/intel/Edge-Software-Provisioner/compare/v2.0...v2.0.1
163+
[2.0.2]: https://github.com/intel/Edge-Software-Provisioner/compare/v2.0.1...v2.0.2
164+
[2.0.3]: https://github.com/intel/Edge-Software-Provisioner/compare/v2.0.2...v2.0.3
165+
[2.5.0]: https://github.com/intel/Edge-Software-Provisioner/compare/v2.0.3...v2.5

‎Jenkinsfile‎

Lines changed: 93 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,93 @@
1+
pipeline {
2+
agent { label 'rbhe' }
3+
stages {
4+
stage('Build') {
5+
environment {
6+
DOCKER_BUILD_ARGS = '--build-arg http_proxy --build-arg https_proxy' // add --no-cache for a clean build
7+
}
8+
steps {
9+
// This really should be pulled out into a script in the source code repo
10+
// like ./ci-build.sh or something similar
11+
sh '''
12+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-aws-cli dockerfiles/aws-cli
13+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-wget dockerfiles/wget
14+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-git dockerfiles/git
15+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-dnsmasq dockerfiles/dnsmasq
16+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-squid dockerfiles/squid
17+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-web dockerfiles/nginx
18+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-gitea dockerfiles/gitea
19+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-qemu dockerfiles/qemu
20+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-smb dockerfiles/smb
21+
22+
# just need to trick the core builder. This image will not run, just needs to be built to be scanned by Snyk
23+
for dir in conf data dockerfiles/core scripts template; do mkdir -p dockerfiles/core/files/${dir}; done
24+
cp ./*.sh dockerfiles/core/files/
25+
cp ./dockerfiles/core/init.sh dockerfiles/core/files/dockerfiles/core/init.sh
26+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-core dockerfiles/core
27+
rm -rf dockerfiles/core/files
28+
29+
# just need to trick the certbot builder. This image will not run, just needs to be built to be scanned by Snyk
30+
mkdir -p dockerfiles/certbot/scripts
31+
docker build ${DOCKER_BUILD_ARGS} -t edgebuilder-certbot dockerfiles/certbot
32+
rm -rf dockerfiles/certbot/scripts
33+
34+
docker images | grep "edgebuilder"
35+
'''
36+
}
37+
}
38+
39+
stage('Static Code Scan') {
40+
when {
41+
expression { env.GIT_BRANCH == 'master' }
42+
}
43+
stages {
44+
stage('Prep Snyk Env') {
45+
steps {
46+
script {
47+
def _files = [
48+
'edgebuilder-aws-cli': 'dockerfiles/aws-cli/Dockerfile',
49+
'edgebuilder-wget': 'dockerfiles/wget/Dockerfile',
50+
'edgebuilder-git': 'dockerfiles/git/Dockerfile',
51+
'edgebuilder-dnsmasq': 'dockerfiles/dnsmasq/Dockerfile',
52+
'edgebuilder-squid': 'dockerfiles/squid/Dockerfile',
53+
'edgebuilder-web': 'dockerfiles/nginx/Dockerfile',
54+
'edgebuilder-gitea': 'dockerfiles/gitea/Dockerfile',
55+
'edgebuilder-qemu': 'dockerfiles/qemu/Dockerfile',
56+
'edgebuilder-smb': 'dockerfiles/smb/Dockerfile',
57+
'edgebuilder-core': 'dockerfiles/core/Dockerfile',
58+
'edgebuilder-certbot': 'dockerfiles/certbot/Dockerfile',
59+
]
60+
61+
env.SNYK_MANIFEST_FILE = _files.collect { k,v -> v }.join(',')
62+
env.SNYK_PROJECT_NAME = _files.collect { k,v -> "${k}-docker" }.join(',')
63+
env.SNYK_DOCKER_IMAGE = _files.collect { k,v -> k }.join(',')
64+
65+
env.SNYK_ALLOW_LONG_PROJECT_NAME = 'true'
66+
env.SNYK_SEVERITY_THRESHOLD_CVE = 'high'
67+
}
68+
}
69+
}
70+
71+
stage('Scan') {
72+
environment {
73+
SCANNERS = 'protex,snyk'
74+
PROJECT_NAME = 'NEX – Container First Architecture'
75+
}
76+
steps {
77+
rbheStaticCodeScan()
78+
}
79+
}
80+
81+
stage('Virus Scan') {
82+
steps {
83+
script {
84+
virusScan {
85+
dir = '.'
86+
}
87+
}
88+
}
89+
}
90+
}
91+
}
92+
}
93+
}

‎LICENSE‎

Lines changed: 9 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,11 @@
1-
The following files describe the licenses that apply to this software:
1+
Copyright © 2022, Intel Corporation
22

3-
licenses/bsd.license
4-
licenses/intel.license
3+
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
54

6-
Any other files with the suffix ".license" also apply to this software.
5+
1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
6+
7+
2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
8+
9+
3. Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
10+
11+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

‎README.md‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ To quickly get started follow the [Quick Installation Guide](#quick-installation
3434

3535
1. [Bootable USB](#bootable-usb)
3636

37+
1. [Dynamic Profile](#dynamic-profile)
38+
3739
## What is it?
3840

3941
The Edge Software Provisioner (ESP) enables ODMs, System Integrators and Developers to automate the installation of a complete operating system and software stack (defined by a Profile) on bare-metal or virtual machines using a "Just-in-Time" provisiong process. The software stack can include software components, middleware, firmware, and applications. Automating this process increases velocity by focusing resources on rapid development, validation of use cases and scalable deployment. ESP simplifies customer adoption through confidence gained validating Profiles. Profiles are cloned and distributed through GitHub containing the human readable prescriptive literature to deploy the complete operating system and software stack. In summary, this a scalable, simple bare metal provisioning process including virtual machine provisioning.
@@ -268,6 +270,7 @@ The following kernel parameters can be added to `conf/config.yml`
268270
* `bootstrap` - RESERVED, do not change
269271
* `ubuntuversion` - Use the Ubuntu release name. Defaults to 'cosmic' release
270272
* `debug` - [TRUE | FALSE] Enables a more verbose output
273+
* `resume` - [TRUE | FALSE] Enables the profile to resume opperation after the last fail. Useful for developing profiles
271274
* `httppath` - RESERVED, do not change
272275
* `kernparam` - Used to pass additional kernel parameters to the targeted system. Example format: kernparam=splash:quiet#enable_gvt:1
273276
* `parttype` - RESERVED, do not change
@@ -467,6 +470,68 @@ under [Installation](#installation)
467470

468471
Type `./makeusb.sh -h` to see othe syntax options.
469472

473+
## Dynamic Profile
474+
475+
### Scope
476+
477+
The Dynamic Profile feature allows ESP to install software on a target system without using ESP Menu selection on systems without a monitor or scale production.
478+
479+
### Profile selection
480+
481+
A json file in the following format is used to associate a profile with either the hardware mac address of at least one of the ethernets or cpu type on the target system. An example json file can be found at `conf/dynamic_profiles.json`, edit this file or place the file on github or http server.
482+
Example JSON File:
483+
484+
```json
485+
{
486+
"hardwares": [
487+
{
488+
"id": "1",
489+
"mac": "AA:BB:CC:11:22:33",
490+
"profile": "Ubuntu_21.04"
491+
},
492+
{
493+
"id": "2",
494+
"cpu": "Intel(R) Xeon(R) CPU D-1557",
495+
"profile": "Ubuntu_20.04_Desktop"
496+
}
497+
]
498+
}
499+
```
500+
501+
**IMPORTANT: The json structure has to follow the structure above. The current implementation relies on these keys. The implemented hardware related information is "macaddress". If others are desired, the implementation needs to be adapted**
502+
503+
### Enabling the Dynamic Profile
504+
505+
Several things need to be done to enable the Dynamic Profile
506+
507+
#### Adapting config.yml
508+
In the config.yml, the following section must be present:
509+
510+
```yaml
511+
dynamic_profile:
512+
enabled: true
513+
url: "https://###SOME_URL###/dynamic_profiles.json" or leave blank to read from conf/dynamic_profiles.json
514+
user: "###USE_IF_HOSTED_ON_GITHUB###" or leave blank
515+
token: "###USE_IF_HOSTED_ON_GITHUB###" or leave blank
516+
517+
```
518+
#### Building and running ESP with Dynamic Profile
519+
If building for the first time please use the following command
520+
```
521+
./build.sh
522+
```
523+
Or if you have already built images you use the following command to update configuration
524+
```
525+
./build.sh -S -P
526+
```
527+
528+
you must run is ESP in dynamic mode in order to support the feature with the following command
529+
```
530+
./run.sh
531+
```
532+
533+
534+
470535
## Known Limitations
471536
472537
* The `conf/config.yml` file must specify ALL values comprehensively, as shown in the `conf/config.sample.yml`. Please use `""` for empty values.

0 commit comments

Comments
 (0)