Skip to content

added simple admission control to area_map. #42

Open
kr8x wants to merge 2 commits into
mainfrom
areaadmission
Open

added simple admission control to area_map. #42
kr8x wants to merge 2 commits into
mainfrom
areaadmission

Conversation

@kr8x

@kr8x kr8x commented May 8, 2026

Copy link
Copy Markdown
Collaborator

script voacapload.sh can set the percentage of map requests to generate maps for REL, TOA or MUF.
100 is no restriction.
Default is no restriction.
0 is allow none.

Requests that do not pass admission control just return a blank map for reduced processor load.

…set the percentage of map requests to generate maps for.
@kr8x

kr8x commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

What the empty map looks like

image

@kr8x

kr8x commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

just realized that I do not need to generate 2 empty maps. will update.

even shorter time to generate a blank map. See updated log below

2026-05-08 17:14:07,140 INFO APP - dispatching path /fetchVOACAPArea.pl
2026-05-08 17:14:07,942 INFO TIMING admission failed: 0.80s
[pid: 37|app: 0|req: 1/1] 172.21.0.2 () {32 vars in 800 bytes} [Fri May 8 17:14:07 2026] GET /fetchVOACAPArea.pl?YEAR=2026&MONTH=5&UTC=17&TXLAT=32.790&TXLNG=-96.770&PATH=0&WATTS=100&WIDTH=5940&HEIGHT=2970&MHZ=21.10&TOA=1.0&MODE=38&TOA=1.0&ANTDEDXCONTROL=3&ANTDEINDEX=1034&ANTDEAZ=90.0&ANTDXINDEX=1034&ANTDXAZ=270 => generated 68766 bytes in 1147 msecs (HTTP/1.1 200) 5 headers in 164 bytes (2 switches on core 0)
172.21.0.2 - - [08/May/2026:17:14:08 +0000] "GET /fetchVOACAPArea.pl?YEAR=2026&MONTH=5&UTC=17&TXLAT=32.790&TXLNG=-96.770&PATH=0&WATTS=100&WIDTH=5940&HEIGHT=2970&MHZ=21.10&TOA=1.0&MODE=38&TOA=1.0&ANTDEDXCONTROL=3&ANTDEINDEX=1034&ANTDEAZ=90.0&ANTDXINDEX=1034&ANTDXAZ=270 HTTP/1.1" 200 68766 "-" "Mojolicious (Perl)"

@kr8x

kr8x commented May 8, 2026

Copy link
Copy Markdown
Collaborator Author

admit 100 percent of offered traffic

$ ./voacapload.sh 100
pushing voacap updates
Successfully copied 2.05kB to voacap-service:/app/allowarea_TOA
Successfully copied 2.05kB to voacap-service:/app/allowarea_MUF
Successfully copied 2.05kB to voacap-service:/app/allowarea_REL

@komacke

komacke commented May 11, 2026

Copy link
Copy Markdown
Collaborator

Thanks for putting this together. But let's take a different approach. I'd like to avoid needing to trigger the shell script.

The latest "main" has nginx.conf rate limiting. At the moment it's 10. It should probably be as high as 50 but let's deal with that later. I like your "black map" idea. Let's serve a black back when the rate limit is exceeded.

I recommend the following:

  1. generate a black map of the requested resolution
  2. put a watermark of "Too many requests" (the 429 error)
  3. save it in /opt/hamclock-backend/cache
  4. if the needed resolution is in cache, serve the cache

@kr8x

kr8x commented May 12, 2026

Copy link
Copy Markdown
Collaborator Author

The path forward as I see it.

  1. build a standalone tool to take the curl output and save separate bmp files day.bmp night.bmp. It handles both concatenated maps and compressed maps.
  2. get 3.10 maps fixed to work out concatenation of map handling. This will add user_agent processing to area_map
  3. build a standalone tool to generate cache of data feeds for all supported widths and concat/zlib. It'll run during server instantiation. In addition, for compressed maps, it has to generate a file that will have the header value for X-2Z-lengths: that has to be sent to keep HamClock happy
  4. when ratelimited, nginx dispatches a lightweight python application that just reads width and user_agent to determine the cache file to send back
  5. for testing purposes we can use curl and -H user_agent and temporally change rate limit to say 1 per minute so two back to back curls will see 1 valid then 1 blank response

@kr8x

kr8x commented May 12, 2026

Copy link
Copy Markdown
Collaborator Author

step 1 complete - see https://github.com/kr8x/hcmapfetch

@kr8x

kr8x commented May 14, 2026

Copy link
Copy Markdown
Collaborator Author

step 2 complete

@kr8x

kr8x commented May 14, 2026

Copy link
Copy Markdown
Collaborator Author

step 3 complete - that's a lot of data

dave@victus:hcmapfetch/429$ ls -l
total 302492
-rw-r--r-- 1 dave dave 3485044 May 14 12:14 429-bmp-1320-660.bin
-rw-r--r-- 1 dave dave 70 May 14 12:14 429-bmp-1320-660.txt
-rw-r--r-- 1 dave dave 7841044 May 14 12:15 429-bmp-1980-990.bin
-rw-r--r-- 1 dave dave 70 May 14 12:15 429-bmp-1980-990.txt
-rw-r--r-- 1 dave dave 13939444 May 14 12:15 429-bmp-2640-1320.bin
-rw-r--r-- 1 dave dave 71 May 14 12:15 429-bmp-2640-1320.txt
-rw-r--r-- 1 dave dave 31363444 May 14 12:16 429-bmp-3960-1980.bin
-rw-r--r-- 1 dave dave 71 May 14 12:16 429-bmp-3960-1980.txt
-rw-r--r-- 1 dave dave 55757044 May 14 12:16 429-bmp-5280-2640.bin
-rw-r--r-- 1 dave dave 71 May 14 12:16 429-bmp-5280-2640.txt
-rw-r--r-- 1 dave dave 70567444 May 14 12:17 429-bmp-5940-2970.bin
-rw-r--r-- 1 dave dave 71 May 14 12:17 429-bmp-5940-2970.txt
-rw-r--r-- 1 dave dave 871444 May 14 12:14 429-bmp-660-330.bin
-rw-r--r-- 1 dave dave 69 May 14 12:14 429-bmp-660-330.txt
-rw-r--r-- 1 dave dave 125453044 May 14 12:17 429-bmp-7920-3960.bin
-rw-r--r-- 1 dave dave 72 May 14 12:17 429-bmp-7920-3960.txt
-rw-r--r-- 1 dave dave 10828 May 14 12:10 429-zlib-1320-660.bin
-rw-r--r-- 1 dave dave 88 May 14 12:10 429-zlib-1320-660.txt
-rw-r--r-- 1 dave dave 15128 May 14 12:10 429-zlib-1980-990.bin
-rw-r--r-- 1 dave dave 88 May 14 12:10 429-zlib-1980-990.txt
-rw-r--r-- 1 dave dave 21218 May 14 12:11 429-zlib-2640-1320.bin
-rw-r--r-- 1 dave dave 90 May 14 12:11 429-zlib-2640-1320.txt
-rw-r--r-- 1 dave dave 38264 May 14 12:12 429-zlib-3960-1980.bin
-rw-r--r-- 1 dave dave 90 May 14 12:12 429-zlib-3960-1980.txt
-rw-r--r-- 1 dave dave 61914 May 14 12:12 429-zlib-5280-2640.bin
-rw-r--r-- 1 dave dave 90 May 14 12:12 429-zlib-5280-2640.txt
-rw-r--r-- 1 dave dave 76362 May 14 12:13 429-zlib-5940-2970.bin
-rw-r--r-- 1 dave dave 90 May 14 12:13 429-zlib-5940-2970.txt
-rw-r--r-- 1 dave dave 8218 May 14 12:09 429-zlib-660-330.bin
-rw-r--r-- 1 dave dave 87 May 14 12:09 429-zlib-660-330.txt
-rw-r--r-- 1 dave dave 129716 May 14 12:13 429-zlib-7920-3960.bin
-rw-r--r-- 1 dave dave 91 May 14 12:13 429-zlib-7920-3960.txt
dave@victus:hcmapfetch/429$ du -h .
296M .
dave@victus:hcmapfetch/429$ du -h 429-bmp-7920-3960.bin
120M 429-bmp-7920-3960.bin
dave@victus:hcmapfetch/429$

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants