dani/django-ram
1
0
Fork 0
mirror of https://github.com/daniviga/django-ram.git synced 2025-08-04 13:17:50 +02:00
Code Issues Packages Projects Releases Wiki Activity

Replace custom python connector with ncat (#42)

Browse Source 
* Replace custom made daemon with nmap-ncat

* Use stderr to log ncat output

* Refresh the branch
This commit is contained in:
Daniele Viganò
2025-01-15 18:30:36 +01:00
committed by GitHub
parent 1e7f72e9ec
commit 90211562f9
15 changed files with 84 additions and 197 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
README.md
View File

@@ -23,7 +23,8 @@ security assesment, pentest, ISO certification, etc.
This project probably doesn't match your needs nor expectations. Be aware. This project probably doesn't match your needs nor expectations. Be aware.
Your model train may also catch fire while using this software. > [!CAUTION]
> Your model train may catch fire while using this software.
Check out [my own instance](https://daniele.mynarrowgauge.org). Check out [my own instance](https://daniele.mynarrowgauge.org).
@@ -49,7 +50,7 @@ It has been developed with:
## Requirements ## Requirements
- Python 3.10+ - Python 3.11+
- A USB port when running Arduino hardware (and adaptors if you have a Mac) - A USB port when running Arduino hardware (and adaptors if you have a Mac)
## Web portal installation ## Web portal installation
@@ -99,43 +100,52 @@ connected via serial port, to the network, allowing commands to be sent via a
TCP socket. A response generated by the DCC++ EX board is sent to all connected clients, TCP socket. A response generated by the DCC++ EX board is sent to all connected clients,
providing synchronization between multiple clients (eg. multiple JMRI instances). providing synchronization between multiple clients (eg. multiple JMRI instances).
Its use is not needed when running DCC++ EX from a [WiFi](https://dcc-ex.com/get-started/wifi-setup.html) capable board (like when Its use is not needed when running DCC++ EX from a [WiFi](https://dcc-ex.com/get-started/wifi-setup.html) capable board, like when
using an ESP8266 module or a [Mega+WiFi board](https://dcc-ex.com/advanced-setup/supported-microcontrollers/wifi-mega.html)). using an ESP8266 module, a [Mega+WiFi board](https://dcc-ex.com/reference/hardware/microcontrollers/wifi-mega.html), or an
[ESP32](https://dcc-ex.com/reference/hardware/microcontrollers/esp32.html) (recommended).
### Customize the settings ### Manual setup
The daemon comes with default settings in `config.ini`. You'll need [namp-ncat](https://nmap.org/ncat/) , and `stty` to setup the serial port.
Settings may need to be customized based on your setup.
> [!IMPORTANT]
> Other variants of `nc` or `ncat` may not work as expected.
Then you can run the following commands:
```bash
$ stty -F /dev/ttyACM0 -echo 115200
$ ncat -n -k -l 2560 </dev/ttyACM0 >/dev/ttyACM0
```
> [!IMPORTANT]
> You'll might need to change the serial port (`/dev/ttyACM0`) to match your board.
> [!NOTE]
> Your user will also need access to the device file, so you might need to add it to the `dialout` group.
### Using containers ### Using containers
```bash ```bash
$ cd daemons $ cd connector
$ podman build -t dcc/net-to-serial . $ podman build -t dcc/connector .
$ podman run --group-add keep-groups --device /dev/ttyACM0 -p 2560:2560 dcc/net-to-serial $ podman run -d --group-add keep-groups --device /dev/ttyACM0:/dev/arduino -p 2560:2560 dcc/connector
```
### Manual setup
```bash
$ cd daemons
$ pip install -r requirements.txt
$ python ./net-to-serial.py
``` ```
### Test with a simulator ### Test with a simulator
A [QEMU AVR based simulator](daemons/simulator/README.md) running DCC++ EX is bundled togheter with the `net-to-serial.py` A [QEMU AVR based simulator](daemons/simulator/README.md) running DCC++ EX is bundled togheter with the connector
daemon into a container. To run it: into a container. To run it:
```bash ```bash
$ cd daemons/simulator $ cd connector/simulator
$ podman build -t dcc/net-to-serial:sim . $ podman build -t dcc/connector:sim .
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/net-to-serial:sim $ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/connector:sim
``` ```
To be continued ... > [!WARNING]
> The simulator is intended for light development and testing purposes only and far from being a complete replacement for a real hardware.
## Screenshots ## Screenshots
@@ -146,15 +156,12 @@ To be continued ...
![Screenshot 2023-09-18 at 21-59-30 RGS 1930s short train - Railroad Assets Manager](https://github.com/daniviga/django-ram/assets/1818657/77f9b7c9-27b3-4a65-bad0-26e9cf77e623) ![Screenshot 2023-09-18 at 21-59-30 RGS 1930s short train - Railroad Assets Manager](https://github.com/daniviga/django-ram/assets/1818657/77f9b7c9-27b3-4a65-bad0-26e9cf77e623)
#### Dark mode #### Dark mode
![Screenshot 2023-09-18 at 21-58-22 Company RGS - Railroad Assets Manager](https://github.com/daniviga/django-ram/assets/1818657/c95697c9-0897-46f4-941c-6092271e4743) ![Screenshot 2023-09-18 at 21-58-22 Company RGS - Railroad Assets Manager](https://github.com/daniviga/django-ram/assets/1818657/c95697c9-0897-46f4-941c-6092271e4743)
--- ---
### Backoffice ### Backoffice
![image](https://user-images.githubusercontent.com/1818657/175789937-3e4970a2-b37d-44c3-8605-62dabe209c65.png) ![image](https://user-images.githubusercontent.com/1818657/175789937-3e4970a2-b37d-44c3-8605-62dabe209c65.png)
@@ -166,8 +173,3 @@ To be continued ...
### Rest API ### Rest API
![image](https://user-images.githubusercontent.com/1818657/180622471-ade06c84-c73b-41d5-a2a7-02a95b2ffc02.png) ![image](https://user-images.githubusercontent.com/1818657/180622471-ade06c84-c73b-41d5-a2a7-02a95b2ffc02.png)

9
connector/Dockerfile Normal file
View File

@@ -0,0 +1,9 @@
FROM alpine:edge
RUN apk add --no-cache coreutils nmap-ncat
EXPOSE 2560/tcp
SHELL ["/bin/ash", "-c"]
CMD stty -F /dev/arduino -echo 115200 && \
ncat -n -k -l 2560 </dev/arduino >/dev/arduino

19
connector/README.md Normal file
View File

@@ -0,0 +1,19 @@
# Use a container to implement a serial to net bridge
This uses `ncat` by [namp](https://nmap.org/ncat/) to bridge a serial port to a network port. The serial port is passed to the Podman command (eg. `/dev/ttyUSB0`) and the network port is `2560`.
> [!IMPORTANT]
> Other variants of `nc` or `ncat` may not work as expected.
## Build and run the container
```bash
$ podman buil -t dcc/bridge .
$ podman run -d --device=/dev/ttyUSB0:/dev/arduino -p 2560:2560 --name dcc-bridge dcc/bridge
```
It can be tested with `telnet`:
```bash
$ telnet localhost 2560
```

BIN
connector/simulator/CommandStation-EX-uno-13488e1.elf Executable file
View File

Binary file not shown.

8
connector/simulator/Dockerfile Normal file
View File

@@ -0,0 +1,8 @@
FROM dcc/bridge
RUN apk update && apk add --no-cache qemu-system-avr \
&& mkdir /io
ADD start.sh /usr/local/bin
ADD CommandStation-EX*.elf /io
ENTRYPOINT ["/usr/local/bin/start.sh"]

13
connector/simulator/README.md Normal file
View File

@@ -0,0 +1,13 @@
# Connector and AVR simulator
> [!WARNING]
> The simulator is intended for light development and testing purposes only and far from being a complete replacement for a real hardware.
`qemu-system-avr` tries to use all the CPU cycles (leaving a CPU core stuck at 100%; limit CPU core usage to 10% via `--cpus 0.1`. It can be adjusted on slower machines.
```bash
$ podman build -t dcc/connector:sim .
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/connector:sim
```
All traffic will be collected on the container's `stderr` for debugging purposes.

4
daemons/simulator/start.sh → connector/simulator/start.sh
View File

@@ -7,7 +7,5 @@ if [ -c /dev/pts/0 ]; then
PTY=1 PTY=1
fi fi
sed -i "s/ttyACM0/pts\/${PTY}/" /opt/dcc/config.ini
qemu-system-avr -machine uno -bios /io/CommandStation-EX*.elf -serial pty -daemonize qemu-system-avr -machine uno -bios /io/CommandStation-EX*.elf -serial pty -daemonize
/opt/dcc/net-to-serial.py ncat -n -k -l 2560 -o /dev/stderr </dev/pts/${PTY} >/dev/pts/${PTY}

9
daemons/Dockerfile
View File

@@ -1,9 +0,0 @@
FROM python:3.11-alpine
RUN mkdir /opt/dcc && pip -q install pyserial
ADD net-to-serial.py config.ini /opt/dcc
RUN python3 -q -m compileall /opt/dcc/net-to-serial.py
EXPOSE 2560/tcp
CMD ["python3", "/opt/dcc/net-to-serial.py"]

3
daemons/README.md
View File

@@ -1,3 +0,0 @@
## DCC++ EX connector
See [README.md](../README.md)

14
daemons/config.ini
View File

@@ -1,14 +0,0 @@
[Daemon]
LogLevel = debug
ListeningIP = 0.0.0.0
ListeningPort = 2560
MaxClients = 10
[Serial]
# UNO
Port = /dev/ttyACM0
# Mega WiFi
# Port = /dev/ttyUSB0
Baudrate = 115200
# Timeout in milliseconds
Timeout = 50

120
daemons/net-to-serial.py
View File

@@ -1,120 +0,0 @@
#!/usr/bin/env python3
import re
import logging
import serial
import asyncio
import configparser
from pathlib import Path
class SerialDaemon:
connected_clients = set()
def __init__(self, config):
self.ser = serial.Serial(
config["Serial"]["Port"],
timeout=int(config["Serial"]["Timeout"]) / 1000,
)
self.ser.baudrate = config["Serial"]["Baudrate"]
self.max_clients = int(config["Daemon"]["MaxClients"])
def __del__(self):
try:
self.ser.close()
except AttributeError:
pass
def __read_serial(self):
"""Serial reader wrapper"""
response = b""
while True:
line = self.ser.read_until()
if not line.strip(): # empty line
break
if line.decode().startswith("<*"):
logging.debug("Serial debug: {}".format(line))
else:
response += line
logging.debug("Serial read: {}".format(response))
return response
def __write_serial(self, data):
"""Serial writer wrapper"""
self.ser.write(data)
async def handle_echo(self, reader, writer):
"""Process a request from socket and return the response"""
logging.info(
"Clients already connected: {} (max: {})".format(
len(self.connected_clients),
self.max_clients,
)
)
addr = writer.get_extra_info("peername")[0]
if len(self.connected_clients) < self.max_clients:
self.connected_clients.add(writer)
while True: # keep connection to client open
data = await reader.read(100)
if not data: # client has disconnected
break
logging.info("Received {} from {}".format(data, addr))
self.__write_serial(data)
response = self.__read_serial()
for client in self.connected_clients:
client.write(response)
await client.drain()
logging.info("Sent: {}".format(response))
self.connected_clients.remove(writer)
else:
logging.warning(
"TooManyClients: client {} disconnected".format(addr)
)
writer.close()
await writer.wait_closed()
async def return_board(self):
"""Return the board signature"""
line = ""
# drain the serial until we are ready to go
self.__write_serial(b"<s>")
while "DCC-EX" not in line:
line = self.__read_serial().decode()
board = re.findall(r"<iDCC-EX.*>", line)[0]
return board
async def main():
config = configparser.ConfigParser()
config.read(
Path(__file__).resolve().parent / "config.ini"
) # mimick os.path.join
logging.basicConfig(level=config["Daemon"]["LogLevel"].upper())
sd = SerialDaemon(config)
server = await asyncio.start_server(
sd.handle_echo,
config["Daemon"]["ListeningIP"],
config["Daemon"]["ListeningPort"],
)
addr = server.sockets[0].getsockname()
logging.info("Serving on {} port {}".format(addr[0], addr[1]))
logging.info(
"Proxying to {} (Baudrate: {}, Timeout: {})".format(
config["Serial"]["Port"],
config["Serial"]["Baudrate"],
config["Serial"]["Timeout"],
)
)
logging.info("Initializing board")
logging.info("Board {} ready".format(await sd.return_board()))
async with server:
await server.serve_forever()
if __name__ == "__main__":
asyncio.run(main())

1
daemons/requirements.txt
View File

@@ -1 +0,0 @@
PySerial

BIN
daemons/simulator/CommandStation-EX-uno-7311f2c.elf
View File

Binary file not shown.

7
daemons/simulator/Dockerfile
View File

@@ -1,7 +0,0 @@
FROM dcc/net-to-serial
RUN apk update && apk add qemu-system-avr && mkdir /io
ADD start.sh /opt/dcc
ADD CommandStation-EX*.elf /io
ENTRYPOINT ["/opt/dcc/start.sh"]

8
daemons/simulator/README.md
View File

@@ -1,8 +0,0 @@
# AVR Simulator
`qemu-system-avr` tries to use all the CPU cicles (leaving a CPU core stuck at 100%; limit CPU core usage to 10% via `--cpus 0.1`. It can be adjusted on slower machines.
```bash
$ podman build -t dcc/net-to-serial:sim .
$ podman run --init --cpus 0.1 -d -p 2560:2560 dcc/net-to-serial:sim
```