The table below provides access to a summary report by year and by shower, and reports
on the trajectories and orbits of matched events.
The Annual and Shower reports provide a summary of the number of meteors detected,
diff --git a/archive/website/templates/searchdialog.js b/archive/website/templates/searchdialog.js
index a53350829..e85d171cf 100644
--- a/archive/website/templates/searchdialog.js
+++ b/archive/website/templates/searchdialog.js
@@ -98,6 +98,7 @@ function myFunc(myObj) {
for (x in myObj) {
console.log(myObj[x]);
var dtarr = myObj[x].split(',');
+ var cams = dtarr[4].replaceAll(";", "; ");
if (dtarr[0] > 0 ){
txt +='
| ';
dt = new Date(dtarr[0]*1000);
@@ -109,7 +110,7 @@ function myFunc(myObj) {
txt += " | ";
txt += dtarr[3];
txt += " | ";
- txt += dtarr[4];
+ txt += cams;
txt += " | ";
errimg = "onerror=\"this.onerror=null;this.src='/img/missing.png';\"";
txt += "![\"Image](\"") ";
diff --git a/archive/website/templates/shwrcsvindex.html b/archive/website/templates/shwrcsvindex.html
index 64a4a87e0..8ea0a6659 100644
--- a/archive/website/templates/shwrcsvindex.html
+++ b/archive/website/templates/shwrcsvindex.html
@@ -48,6 +48,7 @@
+
The data here are released under the CC BY 4.0 license, so if you are using the data
whether for scientific or other purposes, your must reference this web site in your work.
diff --git a/archive/website/templates/statreportindex.html b/archive/website/templates/statreportindex.html
index a0fdae7fe..b62bd482a 100644
--- a/archive/website/templates/statreportindex.html
+++ b/archive/website/templates/statreportindex.html
@@ -46,7 +46,7 @@
-
+
The table below provides access to a summary report by station.
The data here are released under the CC BY 4.0 license, so if you are using the data
diff --git a/tests/test_apis.py b/tests/test_apis.py
index 7c81fe892..4394b5d16 100644
--- a/tests/test_apis.py
+++ b/tests/test_apis.py
@@ -172,7 +172,7 @@ def test_getECSV():
apiurl='https://api.ukmeteors.co.uk/getecsv?stat={}&dt={}'
res = requests.get(apiurl.format(stationID, dateStr))
assert res.status_code == 200
- assert res.text.split()[-1] == 'time'
+ assert res.text.split()[-1] == '593.580'
stationID = 'UK0080'
dateStr = '2024-02-25T20:02:00.1'
@@ -188,6 +188,6 @@ def test_getECSVbadDate():
dateStr = '2024-02-25T20:02:'
apiurl='https://api.ukmeteors.co.uk/getecsv?stat={}&dt={}'
res = requests.get(apiurl.format(stationID, dateStr))
- assert res.status_code == 200
- assert res.text.split()[-1] == 'format'
+ assert res.status_code == 502
+ #assert res.text.split()[-1] == 'format'
return
diff --git a/usermgmt/README.md b/usermgmt/README.md
deleted file mode 100644
index 4bb751c08..000000000
--- a/usermgmt/README.md
+++ /dev/null
@@ -1,64 +0,0 @@
-# UKMDA User Management
-
-This folder contains the scripts used to add/maintain new cameras to the network.
-
-## Basic Principles
-A camera consists of an RMS ID, location and pointing direction.
-
-Each _location_ is assigned a unique *AWS* user with credentials that grant suitable permissions to the S3 storage. For example, a contributor in Toytown might be assigned a unique AWS user "Toytown".
-
-AWS credentials are not shared between locations. If a location's credentials are compromised, they can be disabled without impacting the rest of the network and new credentials can then be issued. The *ukmon-pitools* toolset is designed so that new credentials will be picked up the next time *refreshTools.sh* is run on the impacted Pi (either at reboot or manually).
-
-Each _camera_ is allocated a unique *Unix* ID. The public ssh key provided by the operator is used to enable *ukmon-pitools* to login and collect configuration information for the camera. For example if the contributor at Toytown had two cameras facing North and South, the cameras might be named "toytown_s" and "toytown_n". Note that camera names are in lower case, must be unique and that SSH keys should not be shared between cameras. If the operator chooses to share keys they do so at their own risk and it means that if the key is compromised, all their cameras will be unable to connect.
-
-
-# User Management tool
-Unix and AWS User creation and configuration are managed by *stationMaint.ps1* which calls a python programme that uses native AWS and Unix libraries to execute the required commands. The tool requires AWS and SSH credentials which are restricted to administrators (see note below).
-
-## Setup
-Prerequisites:
-* anaconda or miniconda
-* An AWS profile with the usermaintenance IAM permissions. The default profile name is *ukmda_maint*.
-* An SSH key thats permissioned to connect to the server. The default key name is *ukmda_admin*.
-
-To install the app, copy the python files, requirements.txt and powershell script to a folder of your choosing, and update the environment variables in the powershell script as needed. Now run the powershell script which will create a suitable Conda environment, install the requirements and launch the tool.
-
-## Adding a new camera
-To add a new camera the camera operator must supply the following:
-* RMS ID (from the RMS .config file)
-* town or village (eg Toytown)
-* approx pointing direction (eg SW)
-* the SSH public key generated by *ukmon-pitools*.
-* Human name and email address of the operator.
-
-RMS ID, pointing direction and SSH key are unique to each camera. Other values will generally be shared
-by other cameras at the same location.
-
-Once the information has been gathered, select Camera/Add and fill in the boxes. Note that the boxes are part-prepopulated with values from whatever row your cursor was on, so that if you're adding another camera at an existing location, you can save a bit of typing.
-
-## Amending a camera
-The location and pointing direction cannot be changed once set. To change these values you'll need to follow the process to move a camera.
-All other values can be amended by selecting the line and updating the values.
-
-## Moving a camera
-To move a camera to a new location, select the row containing the camera then select Camera/Move. After you fill in any new information, a new unix and AWS user will be created and the configuration files for the old user will be updated. The old camera must then be marked disabled as explained next.
-
-IMPORTANT NOTE: the camera owner should NOT make any changes - the system will automatically update their configuration.
-
-## Disabling a camera
-To disable a camera, change the Active column from 1 to today's date in YYYYMMDD format eg 20220715. This removes the camera from current reporting, but retains the details for any historical reporting.
-
-# Disabling a location
-To disable a location, we revoke the corresponding AWS user's keys, and if necessary, delete the AWS user and Unix ID. No other user will be affected. This is not currently managed by the UI as it is a drastic action that we do not want to carry out accidentally!
-
-# Permissions Needed
-This tool requires two sets of permissions:
-* an AWS profile with permissions to add/amend/delete users in IAM. This is required to manage the AWS roles.
-* an SSH key with permissions to login to the Batch server. This is required to manage the Unix accounts.
-
-# Parameters
-The powershell script lists the parameters that need to be set, such as AWS profile to use, the buckets and so on. The ansible YAML script sets these values.
-
-
-# Copyright
-All code Copyright (C) 2018-2023 Mark McIntyre
\ No newline at end of file
diff --git a/usermgmt/build.ps1 b/usermgmt/build.ps1
deleted file mode 100644
index 325846018..000000000
--- a/usermgmt/build.ps1
+++ /dev/null
@@ -1 +0,0 @@
-pyinstaller .\stationMaint2.py --noconsole --onefile --windowed --icon .\camera.ico --clean
diff --git a/usermgmt/deploy-local.yml b/usermgmt/deploy-local.yml
deleted file mode 100644
index 76677f582..000000000
--- a/usermgmt/deploy-local.yml
+++ /dev/null
@@ -1,76 +0,0 @@
----
-- hosts: ukmonhelper2
- gather_facts: no
- tasks:
- - name: import dev variables
- include_vars: ./dev-vars.yml
- tags: dev
- - name: import prod variables
- include_vars: ./prod-vars.yml
- tags: prod
- - name: Ensures {{windestdir}} exists
- file: path={{windestdir}} state=directory
- delegate_to: localhost
- tags: [dev,prod]
- - name: Ensures {{windestdir}}/caminfo exists
- file: path={{windestdir}}/caminfo state=directory
- delegate_to: localhost
- tags: [dev,prod]
-
- - name: build exe
- ansible.builtin.shell: |
- conda activate ukmon-admin
- cd windows
- pyinstaller ./stationMaint2.py --noconsole --onefile --windowed --icon .\camera.ico
- args:
- executable: pwsh.exe
- delegate_to: localhost
- register: outdata
- tags: [dev,prod]
-
- - name: print debug
- debug:
- var: outdata.stdout
- - name: Copy files
- copy: src={{ item.src }} dest={{ item.dest }} mode={{ item.mode }}
- delegate_to: localhost
- tags: [dev,prod]
- with_items:
- - {src: '{{winsrcdir}}/dist/StationMaint2.exe', dest: '{{windestdir}}/', mode: '754', backup: yes, directory_mode: no }
- - {src: '{{winsrcdir}}/camera.ico', dest: '{{windestdir}}/', mode: '644', backup: yes, directory_mode: no }
- - {src: '{{winsrcdir}}/stationmaint.cfg', dest: '{{windestdir}}/', mode: '644', backup: yes, directory_mode: no }
- - name: update source bucket
- lineinfile:
- path: '{{windestdir}}/stationmaint.cfg'
- regexp: 'SRCBUCKET='
- line: 'SRCBUCKET={{srcbucket}}'
- delegate_to: localhost
- tags: [dev,prod]
- - name: update live bucket
- lineinfile:
- path: '{{windestdir}}/stationmaint.cfg'
- regexp: 'LIVEBUCKET='
- line: 'LIVEBUCKET={{livebucket}}'
- delegate_to: localhost
- tags: [dev,prod]
- - name: update web bucket
- lineinfile:
- path: '{{windestdir}}/stationmaint.cfg'
- regexp: 'WEBSITEBUCKET='
- line: 'WEBSITEBUCKET={{websitebucket}}'
- delegate_to: localhost
- tags: [dev,prod]
- - name: update helper ipaddress
- lineinfile:
- path: '{{windestdir}}/stationmaint.cfg'
- regexp: 'HELPERIP='
- line: 'HELPERIP={{serverip}}'
- delegate_to: localhost
- tags: [dev,prod]
- - name: update helper remotedir
- lineinfile:
- path: '{{windestdir}}/stationmaint.cfg'
- regexp: 'REMOTEDIR='
- line: 'REMOTEDIR={{remotedir}}'
- delegate_to: localhost
- tags: [dev,prod]
diff --git a/usermgmt/dev-vars.yml b/usermgmt/dev-vars.yml
deleted file mode 100644
index 5dfce2f67..000000000
--- a/usermgmt/dev-vars.yml
+++ /dev/null
@@ -1,11 +0,0 @@
- winsrcdir: /mnt/e/dev/meteorhunting/ukmda-dataprocessing/usermgmt/windows
- windestdir: /mnt/e/dev/meteorhunting/testing/usermgmt
- destdir: /home/ec2-user/dev/keymgmt
- srcbucket: mjmm-ukmon-shared
- livebucket: mjmm-ukmon-live
- websitebucket: mjmm-ukmonarchive.co.uk
- liveprofile: default
- archprofile: ukmda_admin
- helper: ukmonhelper2
- serverip: "3.11.55.160"
- remotedir: /home/ec2-user/dev/data
\ No newline at end of file
diff --git a/usermgmt/prod-vars.yml b/usermgmt/prod-vars.yml
deleted file mode 100644
index 63b88f793..000000000
--- a/usermgmt/prod-vars.yml
+++ /dev/null
@@ -1,9 +0,0 @@
- winsrcdir: /mnt/e/dev/meteorhunting/ukmda-dataprocessing/usermgmt/windows
- windestdir: /mnt/f/videos/MeteorCam/usermgmt
- destdir: /home/ec2-user/keymgmt
- srcbucket: ukmda-shared
- livebucket: ukmda-live
- websitebucket: ukmda-website
- helper: ukmonhelper2
- serverip: "3.11.55.160"
- remotedir: /home/ec2-user/prod/data
\ No newline at end of file
diff --git a/usermgmt/server/ukmon.ini b/usermgmt/server/ukmon.ini
deleted file mode 100644
index 59c2a1061..000000000
--- a/usermgmt/server/ukmon.ini
+++ /dev/null
@@ -1,5 +0,0 @@
-# config data for this station
-export LOCATION=STATIONLOCATION
-export UKMONHELPER=3.11.55.160
-export UKMONKEY=~/.ssh/ukmon
-export RMSCFG=~/source/RMS/.config
diff --git a/usermgmt/windows/camera.ico b/usermgmt/windows/camera.ico
deleted file mode 100644
index 33fcafac1..000000000
Binary files a/usermgmt/windows/camera.ico and /dev/null differ
diff --git a/usermgmt/windows/release_notes.md b/usermgmt/windows/release_notes.md
deleted file mode 100644
index de9a86dd7..000000000
--- a/usermgmt/windows/release_notes.md
+++ /dev/null
@@ -1,18 +0,0 @@
-# The UKMON Station Management Tool
-
-This application is used to add, modify and disable RMS stations linked to UKMON. The app is intended for use only by the UKMON committee members responsible for managing users.
-
-## Before Installation
-First you should contact me, Mark McIntyre. You will need to send me your ssh public key, and i will then create a userid for you on our server and send you the details.
-
-## Installation
-Download the zip file and expand it to a location of your choice, for example `c:\programdata\ukmon`
-Create an SSH keypair.
-Edit the config file and add the userid i provided, plus the full path to the ssh private key.
-Save the config file and run the app.
-
-## Use
-The menus should be pretty self-explanatory.
-
-## Release Notes
-2024.04.2 - first release.
\ No newline at end of file
diff --git a/usermgmt/windows/requirements.txt b/usermgmt/windows/requirements.txt
deleted file mode 100644
index 3894e9b5f..000000000
--- a/usermgmt/windows/requirements.txt
+++ /dev/null
@@ -1,8 +0,0 @@
-# Copyright (C) 2018-2023 Mark McIntyre
-boto3
-openpyxl
-pandas
-scp
-tksheet
-paramiko
-pyinstaller
diff --git a/usermgmt/windows/stationMaint.ps1 b/usermgmt/windows/stationMaint.ps1
deleted file mode 100644
index 28eeee7ef..000000000
--- a/usermgmt/windows/stationMaint.ps1
+++ /dev/null
@@ -1,18 +0,0 @@
-# Copyright (C) 2018-2023 Mark McIntyre
-push-Location $psscriptroot
-
-# create conda env if not aleady there
-$isadm=(conda env list | select-string "ukmon-admin")
-if ($isadm.count -eq 0) {
- conda env create -n ukmon-admin python=3.8
- pip install -r requirements.txt
-}
-conda activate ukmon-admin
-
-bash -c "rsync -avz --delete ukmonhelper2:keymgmt/sshkeys/ ./sshkeys"
-bash -c "rsync -avz --delete ukmonhelper2:keymgmt/sshkeys/ ./keys"
-bash -c "rsync -avz --delete ukmonhelper2:keymgmt/sshkeys/ ./csvkeys"
-bash -c "rsync -avz --delete ukmonhelper2:keymgmt/sshkeys/ ./inifs"
-
-python stationMaint2.py
-Pop-Location
\ No newline at end of file
diff --git a/usermgmt/windows/stationMaint2.py b/usermgmt/windows/stationMaint2.py
deleted file mode 100644
index 5e5c8a8c8..000000000
--- a/usermgmt/windows/stationMaint2.py
+++ /dev/null
@@ -1,946 +0,0 @@
-# Copyright (C) 2018-2023 Mark McIntyre
-
-from tksheet import Sheet
-import tkinter as tk
-from tkinter import ttk
-from tkinter import Frame, Menu
-from tkinter import simpledialog
-import tkinter.messagebox as tkMessageBox
-from tkinter.filedialog import askopenfilename
-import boto3
-import os
-import sys
-import paramiko
-import json
-import time
-import datetime
-import tempfile
-from scp import SCPClient
-from boto3.dynamodb.conditions import Key
-import pandas as pd
-from configparser import ConfigParser
-import logging
-import logging.handlers
-
-
-log = logging.getLogger("logger")
-
-
-def loadConfig(cfgdir):
- cfgfile = os.path.join(cfgdir, 'stationmaint.cfg')
- cfg = ConfigParser()
- if not os.path.isfile(cfgfile):
- tkMessageBox.showinfo('Warning', f'config file {cfgfile} not found')
- exit(0)
- cfg.read(cfgfile)
- server = cfg['helper']['helperip']
- user = cfg['helper']['user']
- keyfile = cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- scpcli = c.open_sftp()
- try:
- handle, tmpfnam = tempfile.mkstemp()
- scpcli.get(f'{user}.csv', tmpfnam)
- except Exception:
- log.warning('unable to find AWS key')
- scpcli.close()
- c.close()
- try:
- lis = open(tmpfnam, 'r').readlines()
- os.close(handle)
- os.remove(tmpfnam)
- key, sec = lis[1].split(',')
- except Exception:
- log.warning('malformed AWS key')
- if key:
- log.info('retrieved key details')
- awskeys = {'key': key.strip(), 'secret': sec.strip()}
- else:
- awskeys = {}
- return cfg, awskeys
-
-
-def addRow(newdata=None, stationid=None, site=None, user=None, email=None, ddb=None,
- direction=None, camtype=None, active=None, createdate=None, tblname='camdetails'):
- '''
- add a row to the CamDetails table
- '''
- if not ddb:
- ddb = boto3.resource('dynamodb', region_name='eu-west-2')
- if not newdata:
- newdata = {'stationid': stationid, 'site': site, 'humanName':user, 'eMail': email,
- 'direction': direction, 'camtype': camtype, 'active': active, 'created': createdate,
- 'oldcode': stationid}
- table = ddb.Table(tblname)
- response = table.put_item(Item=newdata)
- if response['ResponseMetadata']['HTTPStatusCode'] != 200:
- log.info(response)
- return
-
-
-def loadLocationDetails(table='camdetails', ddb=None, loadall=False):
- if not ddb:
- ddb = boto3.resource('dynamodb', region_name='eu-west-2')
- table = ddb.Table(table)
- res = table.scan()
- # strictly, should check for LastEvaluatedKey here, in case there was more than 1MB of data,
- # however that equates to around 30,000 users which i hope we never have...
- values = res.get('Items', [])
- camdets = pd.DataFrame(values)
- camdets.sort_values(by=['stationid'], inplace=True)
- if not loadall:
- camdets.dropna(inplace=True, subset=['eMail','humanName','site'])
- camdets['camtype'] = camdets['camtype'].astype(int)
- camdets['active'] = camdets['active'].astype(int)
- return camdets
-
-
-def findLocationInfo(srchstring, ddb=None, statdets=None):
- if statdets is None:
- statdets = loadLocationDetails(ddb=ddb)
- statdets = statdets[statdets.active==1]
- s1 = statdets[statdets.stationid.str.contains(srchstring)]
- s2 = statdets[statdets.eMail.str.contains(srchstring)]
- s3 = statdets[statdets.humanName.str.contains(srchstring)]
- s4 = statdets[statdets.site.str.contains(srchstring)]
- srchres = pd.concat([s1, s2, s3, s4])
- srchres.drop(columns=['oldcode','active','camtype'], inplace=True)
- return srchres
-
-
-def cameraExists(stationid=None, location=None, direction=None, ddb=None, statdets=None):
- if statdets is None:
- statdets = loadLocationDetails(ddb=ddb)
- statdets = statdets[statdets.active==1]
- if stationid:
- if len(statdets[statdets.stationid == stationid]) > 0:
- return True
- if location and direction:
- s1 = statdets[statdets.site == location]
- if len(s1) > 0:
- if len(s1[s1.direction == direction]) > 0:
- return True
- return False
-
-
-def dumpCamTable(outdir, statdets=None, ddb=None, exportmindets=False):
- if statdets is None:
- statdets = loadLocationDetails(ddb=ddb)
- statdets = statdets[statdets.active==1]
- if exportmindets:
- statdets = statdets[['stationid', 'eMail']]
- statdets.to_csv(os.path.join(outdir,'camtable.csv'), index=False)
-
-
-def getCamUpdateDate(camid, ddb=None):
- if not ddb:
- ddb = boto3.resource('dynamodb', region_name='eu-west-2')
- table = ddb.Table('LiveBrightness')
- resp = table.query(KeyConditionExpression=Key('camid').eq(camid),
- IndexName = 'camid-CaptureNight-index',
- ScanIndexForward=False,
- Limit=1,
- Select='SPECIFIC_ATTRIBUTES',
- ProjectionExpression='CaptureNight')
- if len(resp['Items']) > 0:
- return int(resp['Items'][0]['CaptureNight'])
- else:
- return 0
-
-
-class srcResBox(tk.Toplevel):
- '''
- A class for the search dialog
- '''
- def __init__(self, title, message):
- tk.Toplevel.__init__(self)
- self.details_expanded = False
- self.title(title)
- self.geometry('600x250')
- self.minsize(700, 250)
- self.maxsize(700, 550)
- self.rowconfigure(0, weight=0)
- self.rowconfigure(1, weight=1)
- self.columnconfigure(0, weight=1)
-
- button_frame = tk.Frame(self)
- button_frame.grid(row=0, column=0, sticky='nsew')
- button_frame.columnconfigure(0, weight=1)
- button_frame.columnconfigure(1, weight=1)
-
- text_frame = tk.Frame(self)
- text_frame.grid(row=1, column=0, padx=(7, 7), pady=(7, 7), sticky='nsew')
- text_frame.rowconfigure(0, weight=1)
- text_frame.columnconfigure(0, weight=1)
-
- self.textbox = tk.Text(text_frame, height=6)
- self.textbox.insert('1.0', message)
- self.textbox.grid(row=0, column=0, sticky='nsew')
- self.geometry('700x550')
- self.details_expanded = True
-
-
-class infoDialog(simpledialog.Dialog):
- '''
- A class to gather or display info on a camera
- '''
- def __init__(self, parent, title, location, user, email, sshkey='', id=''):
- self.data = []
- self.data.append(id)
- self.data.append(location)
- self.data.append('')
- self.data.append(user)
- self.data.append(email)
- self.data.append(sshkey)
- self.parent = parent
-
- super().__init__(parent, title)
-
- def body(self, frame):
- self.camid_label = tk.Label(frame, width=25, text="RMS ID")
- self.camid_label.pack()
- self.camid_box = tk.Entry(frame, width=25)
- self.camid_box.insert(tk.END, self.data[0])
- self.camid_box.pack()
-
- self.location_label = tk.Label(frame, width=25, text="Location")
- self.location_label.pack()
- self.location_box = tk.Entry(frame, width=25)
- self.location_box.insert(tk.END, self.data[1])
- self.location_box.pack()
-
- self.direction_label = tk.Label(frame, width=25, text="direction")
- self.direction_label.pack()
- self.direction_box = tk.Entry(frame, width=25)
- self.direction_box.pack()
-
- self.ownername_label = tk.Label(frame, width=25, text="owner name")
- self.ownername_label.pack()
- self.ownername_box = tk.Entry(frame, width=25)
- self.ownername_box.insert(tk.END, self.data[3])
- self.ownername_box.pack()
-
- self.email_label = tk.Label(frame, width=25, text="email address")
- self.email_label.pack()
- self.email_box = tk.Entry(frame, width=25)
- self.email_box.insert(tk.END, self.data[4])
- self.email_box.pack()
-
- self.sshkey_label = tk.Label(frame, width=25, text="SSH key")
- self.sshkey_label.pack()
- self.sshkey_box = tk.Entry(frame, width=50)
- self.sshkey_box.insert(tk.END, self.data[5])
- self.sshkey_box.pack()
-
- def ok_pressed(self):
- self.data[0] = self.camid_box.get().strip()
- self.data[1] = self.location_box.get().strip().capitalize()
- self.data[2] = self.direction_box.get().strip().upper()
- self.data[3] = self.ownername_box.get().strip()
- self.data[4] = self.email_box.get().strip()
- self.data[5] = self.sshkey_box.get().strip()
- if cameraExists(location=self.data[1],direction=self.data[2], statdets=self.parent.stationdetails):
- msg = f'{self.data[1]}_{self.data[2]} already exists'
- tk.messagebox.showinfo(title="Information", message=msg)
- else:
- self.destroy()
-
- def cancel_pressed(self):
- self.data[0] = ''
- self.destroy()
-
- def buttonbox(self):
- self.ok_button = tk.Button(self, text='OK', width=5, command=self.ok_pressed)
- self.ok_button.pack(side="left")
- cancel_button = tk.Button(self, text='Cancel', width=5, command=self.cancel_pressed)
- cancel_button.pack(side="right")
- self.bind("", lambda event: self.ok_pressed())
- self.bind("", lambda event: self.cancel_pressed())
-
-
-class statOwnerDialog(simpledialog.Dialog):
- '''
- A class to display station ownership data
- '''
- def __init__(self, parent):
- self.stationdetails = parent.stationdetails
- self.parent = parent
- super().__init__(parent, 'Owner Info')
-
- def body(self, frame):
- columns = ('#1','#2','#3','#4')
- tree = ttk.Treeview(frame, columns=columns, show='headings')
- tree.heading('#1', text='camid')
- tree.heading('#2', text='site')
- tree.heading('#3', text='email')
- tree.heading('#4', text='humanName')
- contacts = []
- for _, li in self.stationdetails.iterrows():
- contacts.append((li['stationid'], li['site'], li['eMail'], li['humanName']))
- for contact in contacts:
- tree.insert('', tk.END, values=contact)
- tree.grid(row=0, column=0, sticky='nsew')
- scrollbar = ttk.Scrollbar(root, orient=tk.VERTICAL, command=tree.yview)
- tree.configure(yscroll=scrollbar.set)
- scrollbar.grid(row=0, column=1, sticky='ns')
- return
-
-
-class CamMaintenance(Frame):
- '''
- The main camera maintenance window class
- '''
- def __init__(self, parent, cfgdir):
- self.parent = parent
- Frame.__init__(self, parent)
-
- self.cfg, awskeys = loadConfig(cfgdir)
- self.conn = boto3.Session(aws_access_key_id=awskeys['key'], aws_secret_access_key=awskeys['secret'])
- self.bucket_name = self.cfg['store']['srcbucket']
-
- os.makedirs('jsonkeys', exist_ok=True)
- os.makedirs('csvkeys', exist_ok=True)
- os.makedirs('users', exist_ok=True)
- os.makedirs('inifs', exist_ok=True)
- os.makedirs('sshkeys', exist_ok=True)
- self.resyncLocalFiles()
-
- self.ddb = self.conn.resource('dynamodb', region_name='eu-west-2')
- try:
- self.stationdetails = loadLocationDetails(ddb=self.ddb)
- except Exception:
- log.info('unable to get operator details - probably wrong AWS profile')
- exit(1)
- tmpdf = self.stationdetails[['site','stationid','direction','camtype','active','humanName','eMail','oldcode','created']]
- tmpdf = tmpdf.sort_values(by=['active','site','stationid'], ascending=[True,True,True])
- self.data = tmpdf.values.tolist()
- self.hdrs = tmpdf.columns.tolist()
-
- self.parent.title('Station Maintenance')
-
- # Make menu
- self.menuBar = Menu(self.parent)
- self.parent.config(menu=self.menuBar)
-
- # File menu
- fileMenu = Menu(self.menuBar, tearoff=0)
- fileMenu.add_command(label="Exit", command=self.on_closing)
-
- camMenu = Menu(self.menuBar, tearoff=0)
- camMenu.add_command(label = "Add Camera", command = self.addCamera)
- camMenu.add_command(label = "Relocate Camera", command = self.moveCamera)
- camMenu.add_command(label = "Deactivate Camera", command = self.delCamera)
- camMenu.add_separator()
- camMenu.add_command(label = "Remove Location", command = self.delOperator)
- camMenu.add_separator()
- camMenu.add_command(label = "Download Platepar", command = self.getPlate)
- camMenu.add_command(label = "Update platepar", command = self.newPlate)
- camMenu.add_separator()
- camMenu.add_command(label = "Update SSH Key", command = self.newSSHKey)
- camMenu.add_command(label = "Update AWS Key", command = self.newAWSKey)
- camMenu.add_separator()
- camMenu.add_command(label = "Check Camera", command = self.checkLastUpdate)
-
- ownMenu = Menu(self.menuBar, tearoff=0)
- ownMenu.add_command(label = "View Owner Data", command = self.viewOwnerData)
- ownMenu.add_command(label = "Search Data", command = self.searchOwnerData)
-
- self.menuBar.add_cascade(label="File", underline=0, menu=fileMenu)
- self.menuBar.add_cascade(label="Camera", underline=0, menu=camMenu)
- self.menuBar.add_cascade(label="Owners", underline=0, menu=ownMenu)
-
- parent.grid_columnconfigure(0, weight = 1)
- parent.grid_rowconfigure(0, weight = 1)
- self.frame = tk.Frame(self)
-
- self.frame.grid_columnconfigure(0, weight = 1)
- self.frame.grid_rowconfigure(0, weight = 1)
- self.sheet = Sheet(self.parent,
- page_up_down_select_row = True,
- expand_sheet_if_paste_too_big = True,
- #empty_vertical = 0,
- column_width = 120,
- startup_select = (0,1,"rows"),
- data = self.data,
- headers = self.hdrs,
- height = 700,
- width = 700)
-
- self.sheet.enable_bindings(("single_select",
- "drag_select",
- "select_all",
- "column_width_resize",
- "double_click_column_resize",
- "row_width_resize",
- "column_height_resize",
- "arrowkeys",
- "row_height_resize",
- "double_click_row_resize",
- "right_click_popup_menu",
- "rc_delete_row",
- "copy",
- "cut",
- "paste",
- "delete",
- "undo",
- "edit_cell"
- ))
- self.sheet.popup_menu_add_command("Sort by this Column", self.columns_sort)
-
- self.frame.grid(row = 1, column = 0, sticky = "nswe")
- self.sheet.grid(row = 0, column = 0, sticky = "nswe")
-
- self.sheet.change_theme("light green")
-
- self.sheet.set_all_column_widths()
-
- #self.sheet.extra_bindings("end_delete_rows", self.end_delete_rows)
- #self.sheet.extra_bindings("column_select", self.column_select)
- self.sheet.extra_bindings([("all_select_events", self.all_extra_bindings),
- ("begin_edit_cell", self.begin_edit_cell),
- ("end_edit_cell", self.end_edit_cell)])
-
- self.sheet.popup_menu_add_command('Sort', self.columns_sort, table_menu = False, index_menu = False)
-
- def hide_columns_right_click(self, event = None):
- currently_displayed = self.sheet.display_columns()
- exclude = set(currently_displayed[c] for c in self.sheet.get_selected_columns())
- indexes = [c for c in currently_displayed if c not in exclude]
- self.sheet.display_columns(indexes = indexes, enable = True, refresh = True)
-
- def all_extra_bindings(self, event):
- pass
-
- def begin_edit_cell(self, event):
- self.oldval = self.data[event[0]][event[1]]
- return self.oldval
-
- def end_edit_cell(self, event):
- #log.info(event)
- if event[3] != self.oldval:
- data = self.data[event[0]]
- data[event[1]] = event[3]
- newdata = {'stationid': data[1], 'site': data[0], 'humanName':data[5], 'eMail': data[6],
- 'direction': data[2], 'camtype': str(data[3]), 'active': int(data[4]), 'oldcode': data[1],
- 'created': data[8]}
- addRow(newdata, ddb=self.ddb)
- return event[3]
-
- def end_delete_rows(self, event):
- pass
-
- def window_resized(self, event):
- pass
-
- def mouse_motion(self, event):
- pass
-
- def deselect(self, event):
- log.info(f'{event} {self.sheet.get_selected_cells()}')
-
- def rc(self, event):
- log.info(event)
-
- def cell_select(self, response):
- pass
-
- def shift_select_cells(self, response):
- pass
-
- def drag_select_cells(self, response):
- pass
-
- def ctrl_a(self, response):
- pass
-
- def row_select(self, response):
- pass
-
- def shift_select_rows(self, response):
- log.info(response)
-
- def drag_select_rows(self, response):
- pass
-
- def columns_sort(self):
- cursel = self.sheet.get_selected_cells()
- col = list(cursel)[0][1]
- log.info(self.hdrs[col])
- pass
-
- def column_select(self, response):
- pass
-
- def shift_select_columns(self, response):
- pass
-
- def drag_select_columns(self, response):
- pass
-
- def on_closing(self):
- outdir = 'stationdetails'
- os.makedirs(outdir, exist_ok=True)
- dumpCamTable(outdir=outdir, statdets=self.stationdetails, exportmindets=False)
- log.info('quitting')
- self.destroy()
- self.parent.quit()
- self.parent.destroy()
-
- def delCamera(self):
- tk.messagebox.showinfo(title="Information", message='To remove a camera, set Active=current date yyyymmdd')
- return
-
- def delOperator(self):
- tk.messagebox.showinfo(title="Information", message='Not implemented yet')
- return
-
- def viewOwnerData(self):
- statOwnerDialog(self)
- return
-
- def searchOwnerData(self):
- srchstring = simpledialog.askstring("Some_Name", "Search String",parent=root)
- srchres = findLocationInfo(srchstring, statdets=self.stationdetails)
- msgtext = ''
- for _, li in srchres.iterrows():
- msgtext = msgtext + f'{li.stationid:10s}{li.site:20s}{li.eMail:30s}{li.humanName:20s}\n'
- srcResBox(title="Results", message=msgtext)
- return
-
- def moveCamera(self):
- self.addCopyCamera(move=True)
- return
-
- def checkLastUpdate(self):
- cursel = self.sheet.get_selected_cells()
- cr = list(cursel)[0][0]
- curdata = self.data[cr]
- camid = curdata[1]
- lastupd = getCamUpdateDate(camid, ddb=self.ddb)
- msg = f'{camid} last sent a live image on {lastupd}'
- tk.messagebox.showinfo(title="Information", message=msg)
- return
-
- def addCamera(self):
- self.addCopyCamera(move=False)
- return
-
- def addCopyCamera(self, move=False):
- cursel = self.sheet.get_selected_cells()
- cr = list(cursel)[0][0]
- curdata = self.data[cr]
- user = curdata[5]
- email = curdata[6]
- if move is True:
- sshkey = self.getSSHkey(curdata[0], curdata[2])
- id = curdata[1]
- title = 'Move Camera'
- oldloc = curdata[0].lower() + '_' + curdata[2].lower()
- created = curdata[7]
- else:
- sshkey = ''
- id = ''
- title = 'Add Camera'
- oldloc = ''
- created = datetime.datetime.now().strftime('%Y%m%d')
- answer = infoDialog(self, title, curdata[0], user, email, sshkey, id)
- if answer.data[0].strip() != '':
- d = answer.data
- rmsid = str(d[0]).upper()
- location = str(d[1]).capitalize()
- cameraname = d[1].lower() + '_' + d[2].lower()
- with open(os.path.join('sshkeys', cameraname + '.pub'), 'w') as outf:
- outf.write(d[5])
- rowdata=[d[1],d[0],d[2],'2','1',d[3],d[4],d[0]]
- self.sheet.insert_row(values=rowdata, idx=0)
- self.addNewAwsUser(location)
- self.createIniFile(cameraname)
- self.addNewUnixUser(location, cameraname, oldloc)
- self.addNewOwner(rmsid, location, str(d[3]), str(d[4]), str(d[2]), '2','1', created)
- log.info('done')
- return
-
- def newSSHKey(self):
- cursel = self.sheet.get_selected_cells()
- cr = list(cursel)[0][0]
- curdata = self.data[cr]
- user,email = self.getUserDetails(self.stationdetails, curdata[1])
- sshkey = ''
- id = ''
- title = 'Update SSH Key'
- answer = infoDialog(self, title, curdata[0], user, email, sshkey, id)
- if answer.data[0].strip() != '':
- d = answer.data
- location = str(d[1]).capitalize()
- cameraname = d[1].lower() + '_' + d[2].lower()
- with open(os.path.join('sshkeys', cameraname + '.pub'), 'w') as outf:
- outf.write(d[5])
- self.addNewUnixUser(location, cameraname, updatemode=2)
- self.datachanged = True
- return
-
- def getPlate(self):
- cursel = self.sheet.get_selected_cells()
- cr = list(cursel)[0][0]
- curdata = self.data[cr]
- ppdir = self.cfg['helper']['platepardir']
- ppdir = os.path.join(ppdir, curdata[1])
- os.makedirs(ppdir, exist_ok=True)
- ppfile = 'platepar_cmn2010.cal'
- site = curdata[0].capitalize()
- camid = curdata[1].upper()
-
- server = self.cfg['helper']['helperip']
- user='ec2-user'
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- scpcli = SCPClient(c.get_transport())
- remotedir = self.cfg['helper']['remotedir']
- remotef=f'{remotedir}/consolidated/platepars/{camid}.json'
- localf = os.path.join(ppdir, ppfile)
- scpcli.get(remotef, localf)
- scpcli.close()
- c.close()
-
- s3 = boto3.client('s3')
- res = s3.list_objects_v2(Bucket=self.bucket_name, Prefix=f'archive/{site}/{camid}/2023/202312/')
- if res['KeyCount'] > 0:
- keys=res['Contents']
- fitsfiles = [x['Key'] for x in keys if 'fits' in x['Key']]
- fitsfiles.sort()
- fitsfiles = fitsfiles[-10:]
- for ff in fitsfiles:
- _, ffname = os.path.split(ff)
- dlf = os.path.join(ppdir, ffname)
- s3.download_file(self.bucket_name, ff, dlf)
- cfgfiles = [x['Key'] for x in keys if '.config' in x['Key']]
- cfgfiles.sort()
- dlf = os.path.join(ppdir,'.config')
- s3.download_file(self.bucket_name, cfgfiles[-1], dlf)
- return
-
- def newPlate(self):
- cursel = self.sheet.get_selected_cells()
- cr = list(cursel)[0][0]
- curdata = self.data[cr]
- ppdir =self.cfg['helper']['platepardir']
- ppdir = os.path.join(ppdir, curdata[1])
- ppfile = 'platepar_cmn2010.cal'
- plate = ''
- title = 'Select New Platepar File'
- plate = askopenfilename(title=title, defaultextension='*.cal',initialdir=ppdir, initialfile=ppfile)
- if plate:
- self.uploadPlatepar(curdata, plate)
- log.info(plate)
- return
-
- def resyncLocalFiles(self):
- server = self.cfg['helper']['helperip']
- user='ec2-user'
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- scpcli = c.open_sftp()
- for fldr in ['csvkeys','inifs','keys','sshkeys']:
- targdir = f'/home/{user}/keymgmt/{fldr}'
- flist = scpcli.listdir_attr(targdir)
- for fil in flist:
- copyme = True
- fname = fil.filename
- localfname = os.path.join(fldr, fname)
- if os.path.isfile(localfname):
- mtime = os.path.getmtime(localfname)
- if fil.st_mtime - mtime > 1:
- copyme = True
- else:
- copyme = False
- if copyme:
- scpcli.get(f'{targdir}/{fname}', localfname)
- print(f'Updated {fname}')
- scpcli.close()
- c.close()
-
- return
-
- def newAWSKey(self):
- cursel = self.sheet.get_selected_cells()
- cr = list(cursel)[0][0]
- curdata = self.data[cr]
- location = curdata[0]
- self.createNewAwsKey(location, self.stationdetails)
-
- def uploadPlatepar(self, camdets, plateparfile):
- server = self.cfg['helper']['helperip']
- user='ec2-user'
- uplfile = f'/tmp/platepar_cmn2010_{camdets[1]}.cal'
- camname = f'{camdets[0]}_{camdets[3]}'.lower()
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- scpcli = SCPClient(c.get_transport())
- scpcli.put(plateparfile, uplfile)
- command = f'sudo mkdir -p /var/sftp/{camname}/platepar/ && sudo chown {camname}:{camname} /var/sftp/{camname}/platepar'
- log.info(f'running {command}')
- _, stdout, stderr = c.exec_command(command, timeout=10)
- for line in iter(stdout.readline, ""):
- log.info(line, end="")
- for line in iter(stderr.readline, ""):
- log.info(line, end="")
- command = f'sudo mv {uplfile} /var/sftp/{camname}/platepar/platepar_cmn2010.cal'
- log.info(f'running {command}')
- _, stdout, stderr = c.exec_command(command, timeout=10)
- for line in iter(stdout.readline, ""):
- log.info(line, end="")
- for line in iter(stderr.readline, ""):
- log.info(line, end="")
- command = f'sudo chown {camname}:{camname} /var/sftp/{camname}/platepar/platepar_cmn2010.cal'
- log.info(f'running {command}')
- _, stdout, stderr = c.exec_command(command, timeout=10)
- for line in iter(stdout.readline, ""):
- log.info(line, end="")
- for line in iter(stderr.readline, ""):
- log.info(line, end="")
- scpcli.close()
- c.close()
- return
-
- def addNewOwner(self, rmsid, location, user, email, direction, camtype, active, created):
- log.info(f'adding new owner {user} with {email} for {rmsid} at {location}')
- newdata = {'stationid': rmsid, 'site': location, 'humanName':user, 'eMail': email,
- 'direction': direction, 'camtype': camtype, 'active': int(active), 'oldcode': rmsid,
- 'created': created}
- addRow(newdata=newdata, ddb=self.ddb)
- return
-
- def addNewAwsUser(self, location):
- log.info(f'adding new location {location} to AWS')
- archkeyf = 'jsonkeys/' + location + '_arch.key'
- archuserdets = 'users/' + location + '_arch.txt'
- archcsvf = os.path.join('csvkeys', location.lower() + '_arch.csv')
- os.makedirs('jsonkeys', exist_ok=True)
- os.makedirs('csvkeys', exist_ok=True)
- os.makedirs('users', exist_ok=True)
-
- iamc = self.conn.client('iam')
- try:
- _ = iamc.get_user(UserName=location)
- log.info('location exists, not adding it')
- archkey = None
- except Exception:
- log.info('new location')
- usr = iamc.create_user(UserName=location)
- with open(archuserdets, 'w') as outf:
- outf.write(str(usr))
- archkey = iamc.create_access_key(UserName=location)
- with open(archkeyf, 'w') as outf:
- outf.write(json.dumps(archkey, indent=4, sort_keys=True, default=str))
- with open(archcsvf,'w') as outf:
- outf.write('Access key ID,Secret access key\n')
- outf.write('{},{}\n'.format(archkey['AccessKey']['AccessKeyId'], archkey['AccessKey']['SecretAccessKey']))
- _ = iamc.add_user_to_group(GroupName='cameras', UserName=location)
- if archkey is not None:
- self.createKeyFile(location)
- return
-
- def createNewAwsKey(self, location, caminfo):
- server = self.cfg['helper']['helperip']
- user='ec2-user'
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- command = f'/home/{user}/keymgmt/updateAwsKey.sh {location} force'
- log.info(f'running {command}')
- _, stdout, stderr = c.exec_command(command, timeout=10)
- for line in iter(stdout.readline, ""):
- log.info(line, end="")
- for line in iter(stderr.readline, ""):
- log.info(line, end="")
- log.info('done')
- c.close()
- return
-
-
- def updateKeyfile(self, caminfo, location):
- server = self.cfg['helper']['platepardir']
- user='ec2-user'
- keyf = os.path.join('jsonkeys', location + '.key')
- currkey = json.load(open(keyf, 'r'))
- archcsvf = os.path.join('csvkeys', location.lower() + '_arch.csv')
- with open(archcsvf,'w') as outf:
- outf.write('Access key ID,Secret access key\n')
- outf.write('{},{}\n'.format(currkey['AccessKey']['AccessKeyId'], currkey['AccessKey']['SecretAccessKey']))
-
- affectedcamlist = caminfo[caminfo.site==location]
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- scpcli = SCPClient(c.get_transport())
- # push the raw keyfile
- scpcli.put(keyf, 'keymgmt/rawkeys/live/')
- scpcli.put(archcsvf, 'keymgmt/rawkeys/csvkeys/')
- scpcli.close()
- c.close()
- for _, cam in affectedcamlist.iterrows():
- cameraname = cam.site.lower() + '_' + cam.site.lower()
- keyfile = os.path.join('sshkeys', cameraname + '.pub')
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = cameraname, pkey = k)
- scpcli = SCPClient(c.get_transport())
- scpcli.put(archcsvf, '.')
- scpcli.close()
- c.close()
-
-
- def getSSHkey(self, loc, dir):
- server= self.cfg['helper']['helperip']
- user='ec2-user'
- tmpdir=os.getenv('TEMP', default='c:/temp')
- cameraname = (loc + '_' + dir).lower()
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- c.connect(hostname = server, username = user, pkey = k)
- scpcli = SCPClient(c.get_transport())
- tmpfil = os.path.join(tmpdir,'./tmp.txt')
- # dont use os.path.join - source is on unix we are on windows!
- scpcli.get(f'keymgmt/sshkeys/{cameraname}.pub', tmpfil)
- scpcli.close()
- c.close()
-
- with open(tmpfil, 'r') as inf:
- lis = inf.readlines()
- #os.remove('./tmp.txt')
- return lis[0].strip()
-
-
- def getUserDetails(self, stationdetails, camid):
- reqdf = stationdetails[stationdetails.stationid == camid]
- if len(reqdf) == 0:
- return '',''
- return reqdf.eMail.iloc[0], reqdf.humanName.iloc[0]
-
-
- def addNewUnixUser(self, location, cameraname, oldcamname='', updatemode=0):
- server = self.cfg['helper']['helperip']
- user='ec2-user'
- log.info(f'adding new Unix user {cameraname}')
- keyfile = self.cfg['helper']['sshkey']
- k = paramiko.RSAKey.from_private_key_file(os.path.expanduser(keyfile))
- c = paramiko.SSHClient()
- c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
- try:
- c.connect(hostname = server, username = user, pkey = k)
- except Exception:
- c.connect(hostname = server+'.', username = user, pkey = k)
- scpcli = SCPClient(c.get_transport())
- scpcli.put(os.path.join('sshkeys', cameraname + '.pub'), 'keymgmt/sshkeys/')
- scpcli.put(os.path.join('keys', location.lower() + '.key'), 'keymgmt/keys/')
- scpcli.put(os.path.join('csvkeys', location + '.csv'), 'keymgmt/csvkeys/')
- scpcli.put(os.path.join('inifs', cameraname + '.ini'), 'keymgmt/inifs/')
- command = f'/home/{user}/keymgmt/addSftpUser.sh {cameraname} {location} {updatemode} {oldcamname}'
- log.info(f'running {command}')
- _, stdout, stderr = c.exec_command(command, timeout=60)
- for line in iter(stdout.readline, ""):
- log.info(line)
- for line in iter(stderr.readline, ""):
- log.info(line)
-
- log.info('done, collecting output')
- infname = os.path.join('keymgmt/inifs/',cameraname + '.ini')
- outfname = os.path.join('./inifs', cameraname + '.ini')
- while os.path.isfile(outfname) is False:
- try:
- time.sleep(3)
- scpcli.get(infname, outfname)
- except Exception:
- continue
- scpcli.close()
- c.close()
- return
-
-
- def createKeyFile(self, location):
- archbucket = self.cfg['store']['srcbucket']
- livebucket = self.cfg['store']['livebucket']
- webbucket = self.cfg['store']['websitebucket']
-
- os.makedirs('keys', exist_ok=True)
- outf = 'keys/' + location.lower() + '.key'
- with open(outf, 'w') as ouf:
- ouf.write('export AWS_DEFAULT_REGION=eu-west-1\n')
- ouf.write(f'export CAMLOC="{location}"\n')
- ouf.write(f'export S3FOLDER="archive/{location}/"\n')
- ouf.write(f'export ARCHBUCKET={archbucket}\n')
- ouf.write(f'export LIVEBUCKET={livebucket}\n')
- ouf.write(f'export WEBBUCKET={webbucket}\n')
- ouf.write('export ARCHREGION=eu-west-2\n')
- ouf.write('export LIVEREGION=eu-west-1\n')
- ouf.write('export MATCHDIR=matches/RMSCorrelate\n')
- return
-
-
- def createIniFile(self, cameraname):
- helperip = self.cfg['helper']['helperip']
- os.makedirs('inifs', exist_ok=True)
- outf = 'inifs/' + cameraname + '.ini'
- with open(outf, 'w') as outf:
- outf.write('# config data for this station\n')
- outf.write(f'export LOCATION={cameraname}\n')
- outf.write(f'export UKMONHELPER={helperip}\n')
- outf.write('export UKMONKEY=~/.ssh/ukmon\n')
- outf.write('export RMSCFG=~/source/RMS/.config\n')
- return
-
-
-def log_timestamp():
- """ Returns timestamp for logging.
- """
- return datetime.datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
-
-
-if __name__ == '__main__':
- # Initialize main window
- dir_ = os.getcwd() #os.path.dirname(os.path.realpath(__file__))
-
- log.setLevel(logging.INFO)
-
- os.makedirs(os.path.join(dir_, 'logs'), exist_ok=True)
- log_file = os.path.join(dir_, 'logs', log_timestamp() + '.log')
- handler = logging.handlers.TimedRotatingFileHandler(log_file, when='D', interval=1) # Log to a different file each day
- handler.setLevel(logging.INFO)
-
- formatter = logging.Formatter(fmt='%(asctime)s-%(levelname)s-%(module)s-line:%(lineno)d - %(message)s', datefmt='%Y/%m/%d %H:%M:%S')
- handler.setFormatter(formatter)
- log.addHandler(handler)
-
- ch = logging.StreamHandler(sys.stdout)
- ch.setLevel(logging.DEBUG)
- formatter = logging.Formatter(fmt='%(asctime)s-%(levelname)s: %(message)s', datefmt='%Y/%m/%d %H:%M:%S')
- ch.setFormatter(formatter)
- log.addHandler(ch)
-
- # Log program start
- log.info("Program start")
-
- root = tk.Tk()
- app = CamMaintenance(root, dir_)
- root.iconbitmap(os.path.join(dir_,'camera.ico'))
- root.protocol("WM_DELETE_WINDOW", app.on_closing)
- app.mainloop()
diff --git a/usermgmt/windows/stationmaint.cfg b/usermgmt/windows/stationmaint.cfg
deleted file mode 100644
index 883b6cd0f..000000000
--- a/usermgmt/windows/stationmaint.cfg
+++ /dev/null
@@ -1,10 +0,0 @@
-[helper]
-USER=adm_mark
-SSHKEY=~/.ssh/ukmonhelper
-HELPERIP=3.11.55.160
-REMOTEDIR=/home/ec2-user/prod/data
-PLATEPARDIR=
-[store]
-SRCBUCKET=ukmda-shared
-LIVEBUCKET=ukmda-live
-WEBSITEBUCKET=ukmda-website
diff --git a/utils/addVidorImg.ps1 b/utils/addVidorImg.ps1
new file mode 100644
index 000000000..7b03fa64d
--- /dev/null
+++ b/utils/addVidorImg.ps1
@@ -0,0 +1,50 @@
+# Copyright (C) 2018-2023 Mark McIntyre
+#
+# powershell script to upload an extra video or image of an event, given a trajectory ID
+
+$loc = get-location
+if ($args.count -lt 1) {
+ write-output "Usage: addVideoorImg.ps1 trajpath"
+ write-output " First create a folder for the trajectory eg 20241113_192754.345_UK"
+ write-output " Copy any additional mp4, mov or jpgs into it, then run this script"
+ exit
+}else {
+ $pth = $args[0]
+}
+set-location $PSScriptRoot
+# load the helper functions
+. .\helperfunctions.ps1
+$ini=get-inicontent analysis.ini
+$fbfolder = $ini['localdata']['fbfolder'].replace('$HOME',$home)
+$fbfolder = $fbfolder + "/fireballs"
+set-location $fbfolder
+$traj = (split-path -leaf $pth)
+
+$yr=$traj.Substring(0,4)
+$ym=$traj.Substring(0,6)
+$ymd=$traj.Substring(0,8)
+$pick=$traj.substring(0,15) + '_trajectory.pickle'
+$zipf=$traj.substring(0,15) + '.zip'
+
+wget https://archive.ukmeteors.co.uk/reports/${yr}/orbits/${ym}/${ymd}/${traj}/${pick} -O ${pth}\${pick}
+
+if ((test-path $pth\jpgs) -eq $false) { mkdir $pth\jpgs }
+if ((test-path $pth\mp4s) -eq $false) { mkdir $pth\mp4s }
+
+move-item $pth\*.jpg $pth\jpgs
+move-item $pth\*.mp4 $pth\mp4s
+
+compress-archive -path $pth\* -DestinationPath $pth\$zipf -force
+
+$apikey=(Get-Content ~/.ssh/fbuploadkey.txt)
+$headers = @{
+ 'Content-type' = 'application/zip'
+ 'Slug' = ${zipf}
+ 'apikey'= ${apikey}
+}
+$url = "https://api.ukmeteors.co.uk/fireballfiles?orbitfile=${zipf}"
+
+invoke-webrequest -uri $url -infile $pth\$zipf -Method PUT -Headers $headers
+
+Write-Output "uploaded $zipf at $(get-date)"
+set-location $loc
diff --git a/utils/analysis.ini b/utils/analysis.ini
new file mode 100644
index 000000000..3d0d32ee3
--- /dev/null
+++ b/utils/analysis.ini
@@ -0,0 +1,29 @@
+# Parameters that define your local configuration
+#
+#
+[pylib]
+pylib=$HOME/OneDrive/dev/ukmda-dataprocessing/archive/ukmon_pylib
+
+[localdata]
+FBFOLDER=$HOME/OneDrive/pictures/meteors
+MTHLYFOLDER=$HOME/OneDrive/pictures/meteors/mthlystacks
+
+[rms]
+RMS_INSTALLED=1
+RMS_LOC=$HOME/OneDrive/dev/RMS
+RMS_ENV=RMS
+
+[wmpl]
+WMPL_INSTALLED=1
+WMPL_LOC=$HOME/OneDrive/dev/WesternMeteorPyLib
+WMPL_ENV=wmpl
+
+[gmn]
+IDFILE=~/.ssh/gmnanalysis
+
+# camid to hostname mappings, yours may be different
+[stations]
+UK0006=uk0006
+UK000F=uk000f
+UK002F=uk002f
+UK001L=uk001l
diff --git a/utils/getBrightest.ps1 b/utils/getBrightest.ps1
new file mode 100644
index 000000000..ba9933b27
--- /dev/null
+++ b/utils/getBrightest.ps1
@@ -0,0 +1,41 @@
+# powershell script to get the 30 best detections from the previous night
+# for sharing on social media
+# note that you will have to go through and delete non-meteors from the images
+
+# requirements - you must clone WesternMeteorPyLib and ukmda-dataprocessing from github
+# to your local development space. I use onedrive\dev for my code - set this location in $codeloc
+
+
+# copyright (c) Mark McIntyre, 2025-
+
+set-location $PSScriptRoot
+
+# load the helper functions
+. .\helperfunctions.ps1
+$ini=get-inicontent analysis.ini
+$bdir = $ini['localdata']['fbfolder'].replace('$HOME',$home)
+$bdir = $bdir + "/brightest"
+set-location $bdir
+
+$outdir = $bdir.replace('\','/')
+
+$wmplloc = $ini['wmpl']['wmpl_loc'].replace('$HOME',$home)
+$repdir = $ini['pylib']['pylib'].replace('$HOME',$home)
+
+$env:pythonpath="$wmplloc"
+Push-Location $repdir
+
+conda activate ukmon-shared
+
+Write-Output "working..."
+if ($args.count -eq 0) {
+ python -c "from reports.findBestMp4s import getBestNSingles;getBestNSingles(numtoget=30,outdir='$outdir')"
+}else{
+ $reqdate = $args[0]
+ python -c "from reports.findBestMp4s import getBestNSingles;getBestNSingles(numtoget=30,outdir='$outdir', reqdate='$reqdate')"
+}
+
+$outdirw = $outdir.replace('/','\')
+explorer "$outdirw"
+
+Pop-Location
\ No newline at end of file
diff --git a/utils/helperfunctions.ps1 b/utils/helperfunctions.ps1
new file mode 100644
index 000000000..b914e30dc
--- /dev/null
+++ b/utils/helperfunctions.ps1
@@ -0,0 +1,125 @@
+# Copyright (C) 2018-2023 Mark McIntyre
+
+# helper functions
+
+Function Get-IniContent {
+ <#
+ .Synopsis
+ Gets the content of an INI file
+
+ .Description
+ Gets the content of an INI file and returns it as a hashtable
+
+ .Notes
+ Author : Oliver Lipkau
+ Blog : http://oliver.lipkau.net/blog/
+ Source : https://github.com/lipkau/PsIni
+ http://gallery.technet.microsoft.com/scriptcenter/ea40c1ef-c856-434b-b8fb-ebd7a76e8d91
+ Version : 1.0 - 2010/03/12 - Initial release
+ 1.1 - 2014/12/11 - Typo (Thx SLDR)
+ Typo (Thx Dave Stiff)
+
+ #Requires -Version 2.0
+
+ .Inputs
+ System.String
+
+ .Outputs
+ System.Collections.Hashtable
+
+ .Parameter FilePath
+ Specifies the path to the input file.
+
+ .Example
+ $FileContent = Get-IniContent "C:\myinifile.ini"
+ -----------
+ Description
+ Saves the content of the c:\myinifile.ini in a hashtable called $FileContent
+
+ .Example
+ $inifilepath | $FileContent = Get-IniContent
+ -----------
+ Description
+ Gets the content of the ini file passed through the pipe into a hashtable called $FileContent
+
+ .Example
+ C:\PS>$FileContent = Get-IniContent "c:\settings.ini"
+ C:\PS>$FileContent["Section"]["Key"]
+ -----------
+ Description
+ Returns the key "Key" of the section "Section" from the C:\settings.ini file
+
+ .Link
+ Out-IniFile
+ #>
+
+ [CmdletBinding()]
+ Param(
+ [ValidateNotNullOrEmpty()]
+ [ValidateScript({(Test-Path $_) -and ((Get-Item $_).Extension -eq ".ini")})]
+ [Parameter(ValueFromPipeline=$True,Mandatory=$True)]
+ [string]$FilePath
+ )
+
+ Begin
+ {Write-Verbose "$($MyInvocation.MyCommand.Name):: Function started"}
+
+ Process
+ {
+ Write-Verbose "$($MyInvocation.MyCommand.Name):: Processing file: $Filepath"
+
+ $ini = @{}
+ switch -regex -file $FilePath
+ {
+ "^\[(.+)\]$" # Section
+ {
+ $section = $matches[1]
+ $ini[$section] = @{}
+ $CommentCount = 0
+ }
+ "^(;.*)$" # Comment
+ {
+ if (!($section))
+ {
+ $section = "No-Section"
+ $ini[$section] = @{}
+ }
+ $value = $matches[1]
+ $CommentCount = $CommentCount + 1
+ $name = "Comment" + $CommentCount
+ $ini[$section][$name] = $value
+ }
+ "(.+?)\s*=\s*(.*)" # Key
+ {
+ if (!($section))
+ {
+ $section = "No-Section"
+ $ini[$section] = @{}
+ }
+ $name,$value = $matches[1..2]
+ $ini[$section][$name] = $value
+ }
+ }
+ Write-Verbose "$($MyInvocation.MyCommand.Name):: Finished Processing file: $FilePath"
+ Return $ini
+ }
+
+ End
+ {Write-Verbose "$($MyInvocation.MyCommand.Name):: Function ended"}
+}
+
+Function Test-CommandExists
+{
+ Param ($command)
+ $oldPreference = $ErrorActionPreference
+ $ErrorActionPreference = ‘stop’
+ try {if(Get-Command $command){RETURN $true}}
+ Catch {Write-Host “$command does not exist”; RETURN $false}
+ Finally {$ErrorActionPreference=$oldPreference}
+} #end function test-CommandExists
+
+Function New-TemporaryFolder {
+ # Make a new folder based upon a TempFileName
+ $T="$($Env:temp)\tmp$([convert]::tostring((get-random 65535),16).padleft(4,'0')).tmp"
+ New-Item -ItemType Directory -Path $T
+}
\ No newline at end of file
diff --git a/utils/uploadOrbit.ps1 b/utils/uploadOrbit.ps1
new file mode 100644
index 000000000..a512a129b
--- /dev/null
+++ b/utils/uploadOrbit.ps1
@@ -0,0 +1,70 @@
+# Copyright (C) 2018-2023 Mark McIntyre
+#
+# powershell script to create a zip file of a solution and upload it
+
+$loc = get-location
+if ($args.count -lt 1) {
+ $curdir = get-location
+}else {
+ $curdir = resolve-path $args[0]
+}
+set-location $PSScriptRoot
+# load the helper functions
+. .\helperfunctions.ps1
+$ini=get-inicontent analysis.ini
+Set-Location $Loc
+
+Add-Type -AssemblyName System.Windows.Forms
+
+conda activate $ini['wmpl']['wmpl_env']
+$wmplloc = $ini['wmpl']['wmpl_loc'].replace('$HOME',$home)
+$env:pythonpath="$wmplloc;$env:pythonpath"
+
+write-output "Find the pickle"
+$picklefile= (Get-ChildItem $curdir\*.pickle -r).fullname
+if ($picklefile -is [Array] -or $picklefile.length -eq 0) {
+ $FileBrowser = New-Object System.Windows.Forms.OpenFileDialog -Property @{ InitialDirectory = $curdir; Filter = 'pickles (*.pickle)|*.pickle'; Title='Select orbit pickle' }
+ $null = $FileBrowser.ShowDialog()
+ $picklefile = $filebrowser.filename
+ if ( $picklefile -eq "" ) {
+ Write-Output "Cancelled"
+ set-location $loc
+ exit
+ }
+}
+write-output "find the orbit full name"
+$srcdir = ((get-item $picklefile).directoryname).replace('\','/')
+$pickname = (get-item $picklefile).name
+$origdir = (python -c "from wmpl.Utils.Pickling import loadPickle;pick = loadPickle('${srcdir}','${pickname}');print(pick.output_dir)")
+$orbname = split-path $origdir -leaf
+if ((test-path $curdir\$orbname) -eq 0 )
+{
+ mkdir -force $curdir\$orbname
+ move-item $picklefile $curdir\$orbname\
+}
+$picklefile = (Get-ChildItem $picklefile -r).fullname
+
+write-output "look for jpgs and mp4s"
+mkdir -force $curdir\jpgs
+$jpgdir = resolve-path $curdir\jpgs
+Move-Item $curdir\*.jpg $jpgdir
+mkdir -force $curdir\mp4s
+$mp4dir = resolve-path $curdir\mp4s
+Move-Item $curdir\*.mp4 $mp4dir
+
+write-output "Creating zip file"
+if ((test-path $env:temp\$orbname\mp4s) -eq 0 ) {mkdir $env:temp\$orbname\mp4s | out-null}
+if ((test-path $env:temp\$orbname\jpgs) -eq 0 ) {mkdir $env:temp\$orbname\jpgs | out-null}
+copy-item $picklefile $env:temp\$orbname
+copy-item $jpgdir\*.jpg $env:temp\$orbname\jpgs
+copy-item $mp4dir\*.mp4 $env:temp\$orbname\mp4s
+if ((test-path $env:temp\$orbname.zip) -eq 1 ) { remove-item $env:temp\$orbname.zip}
+compress-archive -destinationpath "$env:temp\$orbname.zip" -path "$env:temp\$orbname\*"
+remove-item $env:temp\$orbname\* -Recurse
+
+write-output "Uploading $orbname.zip"
+curl -X PUT -H "Content-Type:application/zip" --data-binary "@$env:temp\$orbname.zip" "https://api.ukmeteors.co.uk/fireballfiles?orbitfile=$orbname.zip"
+move-item $env:temp\$orbname.zip $curdir -Force
+Write-Output "uploaded at $(get-date)"
+set-location $loc
+conda deactivate
|