mirror of
https://github.com/daniviga/django-ram.git
synced 2025-08-03 20:57:50 +02:00
Replace custom python connector with ncat (#42)
* Replace custom made daemon with nmap-ncat * Use stderr to log ncat output * Refresh the branch
This commit is contained in:
GitHub
66
README.md
66
README.md
@@ -23,7 +23,8 @@ security assesment, pentest, ISO certification, etc.
|
||||
|
||||
This project probably doesn't match your needs nor expectations. Be aware.
|
||||
|
||||
Your model train may also catch fire while using this software.
|
||||
> [!CAUTION]
|
||||
> Your model train may catch fire while using this software.
|
||||
|
||||
Check out [my own instance](https://daniele.mynarrowgauge.org).
|
||||
|
||||
@@ -49,7 +50,7 @@ It has been developed with:
|
||||
|
||||
## Requirements
|
||||
|
||||
- Python 3.10+
|
||||
- Python 3.11+
|
||||
- A USB port when running Arduino hardware (and adaptors if you have a Mac)
|
||||
|
||||
## Web portal installation
|
||||
@@ -99,43 +100,52 @@ connected via serial port, to the network, allowing commands to be sent via a
|
||||
TCP socket. A response generated by the DCC++ EX board is sent to all connected clients,
|
||||
providing synchronization between multiple clients (eg. multiple JMRI instances).
|
||||
|
||||
Its use is not needed when running DCC++ EX from a [WiFi](https://dcc-ex.com/get-started/wifi-setup.html) capable board (like when
|
||||
using an ESP8266 module or a [Mega+WiFi board](https://dcc-ex.com/advanced-setup/supported-microcontrollers/wifi-mega.html)).
|
||||
Its use is not needed when running DCC++ EX from a [WiFi](https://dcc-ex.com/get-started/wifi-setup.html) capable board, like when
|
||||
using an ESP8266 module, a [Mega+WiFi board](https://dcc-ex.com/reference/hardware/microcontrollers/wifi-mega.html), or an
|
||||
[ESP32](https://dcc-ex.com/reference/hardware/microcontrollers/esp32.html) (recommended).
|
||||
|
||||
### Customize the settings
|
||||
### Manual setup
|
||||
|
||||
The daemon comes with default settings in `config.ini`.
|
||||
Settings may need to be customized based on your setup.
|
||||
You'll need [namp-ncat](https://nmap.org/ncat/) , and `stty` to setup the serial port.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Other variants of `nc` or `ncat` may not work as expected.
|
||||
|
||||
Then you can run the following commands:
|
||||
|
||||
```bash
|
||||
$ stty -F /dev/ttyACM0 -echo 115200
|
||||
$ ncat -n -k -l 2560 </dev/ttyACM0 >/dev/ttyACM0
|
||||
```
|
||||
|
||||
> [!IMPORTANT]
|
||||
> You'll might need to change the serial port (`/dev/ttyACM0`) to match your board.
|
||||
|
||||
> [!NOTE]
|
||||
> Your user will also need access to the device file, so you might need to add it to the `dialout` group.
|
||||
|
||||
|
||||
### Using containers
|
||||
|
||||
```bash
|
||||
$ cd daemons
|
||||
$ podman build -t dcc/net-to-serial .
|
||||
$ podman run --group-add keep-groups --device /dev/ttyACM0 -p 2560:2560 dcc/net-to-serial
|
||||
```
|
||||
|
||||
### Manual setup
|
||||
|
||||
```bash
|
||||
$ cd daemons
|
||||
$ pip install -r requirements.txt
|
||||
$ python ./net-to-serial.py
|
||||
$ cd connector
|
||||
$ podman build -t dcc/connector .
|
||||
$ podman run -d --group-add keep-groups --device /dev/ttyACM0:/dev/arduino -p 2560:2560 dcc/connector
|
||||
```
|
||||
|
||||
### Test with a simulator
|
||||
|
||||
A [QEMU AVR based simulator](daemons/simulator/README.md) running DCC++ EX is bundled togheter with the `net-to-serial.py`
|
||||
daemon into a container. To run it:
|
||||
A [QEMU AVR based simulator](daemons/simulator/README.md) running DCC++ EX is bundled togheter with the connector
|
||||
into a container. To run it:
|
||||
|
||||
```bash
|
||||
$ cd daemons/simulator
|
||||
$ podman build -t dcc/net-to-serial:sim .
|
||||
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/net-to-serial:sim
|
||||
$ cd connector/simulator
|
||||
$ podman build -t dcc/connector:sim .
|
||||
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/connector:sim
|
||||
```
|
||||
|
||||
To be continued ...
|
||||
> [!WARNING]
|
||||
> The simulator is intended for light development and testing purposes only and far from being a complete replacement for a real hardware.
|
||||
|
||||
## Screenshots
|
||||
|
||||
@@ -146,15 +156,12 @@ To be continued ...
|
||||
|
||||
|
||||
|
||||
|
||||
#### Dark mode
|
||||
|
||||
|
||||
|
||||
---
|
||||
|
||||
|
||||
|
||||
### Backoffice
|
||||
|
||||
|
||||
@@ -166,8 +173,3 @@ To be continued ...
|
||||
### Rest API
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
9
connector/Dockerfile
Normal file
9
connector/Dockerfile
Normal file
@@ -0,0 +1,9 @@
|
||||
FROM alpine:edge
|
||||
|
||||
RUN apk add --no-cache coreutils nmap-ncat
|
||||
|
||||
EXPOSE 2560/tcp
|
||||
|
||||
SHELL ["/bin/ash", "-c"]
|
||||
CMD stty -F /dev/arduino -echo 115200 && \
|
||||
ncat -n -k -l 2560 </dev/arduino >/dev/arduino
|
||||
19
connector/README.md
Normal file
19
connector/README.md
Normal file
@@ -0,0 +1,19 @@
|
||||
# Use a container to implement a serial to net bridge
|
||||
|
||||
This uses `ncat` by [namp](https://nmap.org/ncat/) to bridge a serial port to a network port. The serial port is passed to the Podman command (eg. `/dev/ttyUSB0`) and the network port is `2560`.
|
||||
|
||||
> [!IMPORTANT]
|
||||
> Other variants of `nc` or `ncat` may not work as expected.
|
||||
|
||||
## Build and run the container
|
||||
|
||||
```bash
|
||||
$ podman buil -t dcc/bridge .
|
||||
$ podman run -d --device=/dev/ttyUSB0:/dev/arduino -p 2560:2560 --name dcc-bridge dcc/bridge
|
||||
```
|
||||
|
||||
It can be tested with `telnet`:
|
||||
|
||||
```bash
|
||||
$ telnet localhost 2560
|
||||
```
|
||||
BIN
connector/simulator/CommandStation-EX-uno-13488e1.elf
Executable file
BIN
connector/simulator/CommandStation-EX-uno-13488e1.elf
Executable file
Binary file not shown.
8
connector/simulator/Dockerfile
Normal file
8
connector/simulator/Dockerfile
Normal file
@@ -0,0 +1,8 @@
|
||||
FROM dcc/bridge
|
||||
|
||||
RUN apk update && apk add --no-cache qemu-system-avr \
|
||||
&& mkdir /io
|
||||
ADD start.sh /usr/local/bin
|
||||
ADD CommandStation-EX*.elf /io
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/start.sh"]
|
||||
13
connector/simulator/README.md
Normal file
13
connector/simulator/README.md
Normal file
@@ -0,0 +1,13 @@
|
||||
# Connector and AVR simulator
|
||||
|
||||
> [!WARNING]
|
||||
> The simulator is intended for light development and testing purposes only and far from being a complete replacement for a real hardware.
|
||||
|
||||
`qemu-system-avr` tries to use all the CPU cycles (leaving a CPU core stuck at 100%; limit CPU core usage to 10% via `--cpus 0.1`. It can be adjusted on slower machines.
|
||||
|
||||
```bash
|
||||
$ podman build -t dcc/connector:sim .
|
||||
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/connector:sim
|
||||
```
|
||||
|
||||
All traffic will be collected on the container's `stderr` for debugging purposes.
|
||||
@@ -7,7 +7,5 @@ if [ -c /dev/pts/0 ]; then
|
||||
PTY=1
|
||||
fi
|
||||
|
||||
sed -i "s/ttyACM0/pts\/${PTY}/" /opt/dcc/config.ini
|
||||
|
||||
qemu-system-avr -machine uno -bios /io/CommandStation-EX*.elf -serial pty -daemonize
|
||||
/opt/dcc/net-to-serial.py
|
||||
ncat -n -k -l 2560 -o /dev/stderr </dev/pts/${PTY} >/dev/pts/${PTY}
|
||||
@@ -1,9 +0,0 @@
|
||||
FROM python:3.11-alpine
|
||||
|
||||
RUN mkdir /opt/dcc && pip -q install pyserial
|
||||
ADD net-to-serial.py config.ini /opt/dcc
|
||||
RUN python3 -q -m compileall /opt/dcc/net-to-serial.py
|
||||
|
||||
EXPOSE 2560/tcp
|
||||
|
||||
CMD ["python3", "/opt/dcc/net-to-serial.py"]
|
||||
@@ -1,3 +0,0 @@
|
||||
## DCC++ EX connector
|
||||
|
||||
See [README.md](../README.md)
|
||||
@@ -1,14 +0,0 @@
|
||||
[Daemon]
|
||||
LogLevel = debug
|
||||
ListeningIP = 0.0.0.0
|
||||
ListeningPort = 2560
|
||||
MaxClients = 10
|
||||
|
||||
[Serial]
|
||||
# UNO
|
||||
Port = /dev/ttyACM0
|
||||
# Mega WiFi
|
||||
# Port = /dev/ttyUSB0
|
||||
Baudrate = 115200
|
||||
# Timeout in milliseconds
|
||||
Timeout = 50
|
||||
@@ -1,120 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
import re
|
||||
import logging
|
||||
import serial
|
||||
import asyncio
|
||||
import configparser
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
class SerialDaemon:
|
||||
connected_clients = set()
|
||||
|
||||
def __init__(self, config):
|
||||
self.ser = serial.Serial(
|
||||
config["Serial"]["Port"],
|
||||
timeout=int(config["Serial"]["Timeout"]) / 1000,
|
||||
)
|
||||
self.ser.baudrate = config["Serial"]["Baudrate"]
|
||||
self.max_clients = int(config["Daemon"]["MaxClients"])
|
||||
|
||||
def __del__(self):
|
||||
try:
|
||||
self.ser.close()
|
||||
except AttributeError:
|
||||
pass
|
||||
|
||||
def __read_serial(self):
|
||||
"""Serial reader wrapper"""
|
||||
response = b""
|
||||
while True:
|
||||
line = self.ser.read_until()
|
||||
if not line.strip(): # empty line
|
||||
break
|
||||
if line.decode().startswith("<*"):
|
||||
logging.debug("Serial debug: {}".format(line))
|
||||
else:
|
||||
response += line
|
||||
logging.debug("Serial read: {}".format(response))
|
||||
|
||||
return response
|
||||
|
||||
def __write_serial(self, data):
|
||||
"""Serial writer wrapper"""
|
||||
self.ser.write(data)
|
||||
|
||||
async def handle_echo(self, reader, writer):
|
||||
"""Process a request from socket and return the response"""
|
||||
logging.info(
|
||||
"Clients already connected: {} (max: {})".format(
|
||||
len(self.connected_clients),
|
||||
self.max_clients,
|
||||
)
|
||||
)
|
||||
|
||||
addr = writer.get_extra_info("peername")[0]
|
||||
if len(self.connected_clients) < self.max_clients:
|
||||
self.connected_clients.add(writer)
|
||||
while True: # keep connection to client open
|
||||
data = await reader.read(100)
|
||||
if not data: # client has disconnected
|
||||
break
|
||||
logging.info("Received {} from {}".format(data, addr))
|
||||
self.__write_serial(data)
|
||||
response = self.__read_serial()
|
||||
for client in self.connected_clients:
|
||||
client.write(response)
|
||||
await client.drain()
|
||||
logging.info("Sent: {}".format(response))
|
||||
self.connected_clients.remove(writer)
|
||||
else:
|
||||
logging.warning(
|
||||
"TooManyClients: client {} disconnected".format(addr)
|
||||
)
|
||||
|
||||
writer.close()
|
||||
await writer.wait_closed()
|
||||
|
||||
async def return_board(self):
|
||||
"""Return the board signature"""
|
||||
line = ""
|
||||
# drain the serial until we are ready to go
|
||||
self.__write_serial(b"<s>")
|
||||
while "DCC-EX" not in line:
|
||||
line = self.__read_serial().decode()
|
||||
board = re.findall(r"<iDCC-EX.*>", line)[0]
|
||||
return board
|
||||
|
||||
|
||||
async def main():
|
||||
config = configparser.ConfigParser()
|
||||
config.read(
|
||||
Path(__file__).resolve().parent / "config.ini"
|
||||
) # mimick os.path.join
|
||||
logging.basicConfig(level=config["Daemon"]["LogLevel"].upper())
|
||||
|
||||
sd = SerialDaemon(config)
|
||||
server = await asyncio.start_server(
|
||||
sd.handle_echo,
|
||||
config["Daemon"]["ListeningIP"],
|
||||
config["Daemon"]["ListeningPort"],
|
||||
)
|
||||
addr = server.sockets[0].getsockname()
|
||||
logging.info("Serving on {} port {}".format(addr[0], addr[1]))
|
||||
logging.info(
|
||||
"Proxying to {} (Baudrate: {}, Timeout: {})".format(
|
||||
config["Serial"]["Port"],
|
||||
config["Serial"]["Baudrate"],
|
||||
config["Serial"]["Timeout"],
|
||||
)
|
||||
)
|
||||
logging.info("Initializing board")
|
||||
logging.info("Board {} ready".format(await sd.return_board()))
|
||||
|
||||
async with server:
|
||||
await server.serve_forever()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -1 +0,0 @@
|
||||
PySerial
|
||||
Binary file not shown.
@@ -1,7 +0,0 @@
|
||||
FROM dcc/net-to-serial
|
||||
|
||||
RUN apk update && apk add qemu-system-avr && mkdir /io
|
||||
ADD start.sh /opt/dcc
|
||||
ADD CommandStation-EX*.elf /io
|
||||
|
||||
ENTRYPOINT ["/opt/dcc/start.sh"]
|
||||
@@ -1,8 +0,0 @@
|
||||
# AVR Simulator
|
||||
|
||||
`qemu-system-avr` tries to use all the CPU cicles (leaving a CPU core stuck at 100%; limit CPU core usage to 10% via `--cpus 0.1`. It can be adjusted on slower machines.
|
||||
|
||||
```bash
|
||||
$ podman build -t dcc/net-to-serial:sim .
|
||||
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/net-to-serial:sim
|
||||
```
|
||||
Reference in New Issue
Block a user