Emulating H3C Networks Outside HCL

h3c homelab gns3 firmware-analysis

My homelab involves devices from pretty different manufacturers (H3C, Fortinet, …), where I have been suffered a lot. It was quite common to see that I became disconnected from the Internet, because of dealing with the network configuration, especially the interoperability between different manufacturers. The good side is that it helps me to pick up various styles of configuration.

If all my devices were aligned by one manufacturer, I could just use their official emulator to redesign my networks in a volatile way. To reduce the waste of time, I really want to emulate my whole networks to facilitate a quick validation. But the bad news is that generic emulators like GNS3 / EVE-NG, never include H3C devices. Currently the only solution for emulating H3C networks is their official HCL, which is very heavy, and only targets at Windows.

While negating the reality is useless, it naturally derives the idea—“jailbreak” the devices outside HCL, make them running in QEMU instead.

Good News: Native i386 Images#

HCL utilizes VirtualBox and runs i386 guests for various devices. There are no proprietary emulators.

H3C S6850

There are 4 virtual disks attached to the guest. Their partitions are listed here:

$ lsblk -f /dev/nbd{0..3}
NAME     FSTYPE FSVER LABEL UUID                                 FSAVAIL FSUSE% MOUNTPOINTS
nbd0                                                                            
└─nbd0p1 vfat   FAT32       1A75-468D                                           
nbd1                                                                            
└─nbd1p1 vfat   FAT32       2C77-DBB6                                           
nbd2                                                                            
├─nbd2p1 vfat   FAT32       D257-0826                                           
└─nbd2p2 vfat   FAT16       D2A2-D748                                           
nbd3                                                                            
├─nbd3p1 vfat   FAT32       D257-0826                                           
└─nbd3p2 vfat   FAT16       D2A2-D748

After inspecting the contents, disks are used for:

  • IDE 0 Master:
    • FAT32: GRUB bootloader
  • IDE 0 Slave:
    • FAT32: Storing .ipe system images
  • IDE 1 Master:
    • FAT32: Flash
    • FAT16: NVRAM
  • IDE 1 Slave:
    • FAT32: Flash
    • FAT16: NVRAM

Bad News: impl !OutOfTheBox#

The images do not work out of the box, what’s even worse is that its way of interaction erases any possibility to make it run on existing network emulators.

DMI Validation#

The original VirtualBox machine definition contains something worthy of note:

1
2
3
4
5
6
7
8
9

<ExtraData>
  <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardProduct" value="id=1 type=2100 slot=0 dev=58 macAddr=32-0f-2e-a1-01-00"/>
  <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardSerial" value=""/>
  <ExtraDataItem name="VBoxInternal/Devices/pcbios/0/Config/DmiBoardVersion" value="mem=512"/>
</ExtraData>
<UART>
  <Port slot="0" enabled="true" IOBase="0x3f8" IRQ="4" path="temp\hcl_wtg_k59f\SerialFile\S6850_1.txt" hostMode="RawFile"/>
  <Port slot="1" enabled="true" IOBase="0x2f8" IRQ="3" server="true" path="\\.\pipe\topo1-device1" hostMode="HostPipe"/>
</UART>

The device image will verify the DMI info in its kernel module system.ko:

Ghidra Showing DMI Verification

The solution is to add the following SMBIOS information when running QEMU

1
2
3
4
5
6

sn=$(echo AQMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwQEBAQFBQ== | base64 -d)
args=(
  -smbios type=0,version='VirtualBox'
  -smbios type=2,product='id=1 type=2100 slot=0 dev=58 macAddr=2a-b3-b4-9a-01-00',serial="$sn",version='mem=512'
)
# plug it in command line using "${args[@]}"

Network adapters for the virtual machines are listed here:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

<Network>
  <Adapter slot="0" enabled="true" MACAddress="08002721ADB4" promiscuousModePolicy="AllowAll">
    <DisabledModes>
      <InternalNetwork name="intnet"/>
      <NATNetwork name="NatNetwork"/>
    </DisabledModes>
    <GenericInterface driver="UDPTunnel">
      <Property name="dest" value="127.0.0.1"/>
      <Property name="dport" value="10000"/>
      <Property name="sport" value="10001"/>
    </GenericInterface>
  </Adapter>
  <Adapter slot="1" enabled="true" MACAddress="00E001021234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
  <Adapter slot="2" enabled="true" MACAddress="00E001031234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
  <Adapter slot="3" enabled="true" MACAddress="00E001041234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
  <Adapter slot="4" enabled="true" MACAddress="00E001051234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
  <Adapter slot="5" enabled="true" MACAddress="00E001061234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
  <Adapter slot="6" enabled="true" MACAddress="00E001071234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
  <Adapter slot="7" enabled="true" MACAddress="00E001081234" cable="false" promiscuousModePolicy="AllowAll" type="82540EM"/>
</Network>

The first NIC is actually a control channel between HCL and the device, for link status update and configuration synchronization. If the first logical Ethernet port of a switch is connected to another switch, HCL will send a UDP datagram to localhost:10001 with the following payload:

0000   41 63 74 69 6f 6e 3d 30 20 49 46 4e 4f 3d 31 20   Action=1 IFNO=1 
0010   4e 49 43 49 44 3d 32 20 50 45 45 52 3d 31 20 44   NICID=2 PEER=1 D
0020   49 46 4e 4f 3d 30 20 44 65 76 54 79 70 65 3d 32   IFNO=0 DevType=2
0030   00 00 00 00 00 00 00 00 00 00 00 00               ............

The fields are used as:

  • Action: 1 for connected event, 0 for disconnected event
  • IFNO: # logical Ethernet port, starting from 1
  • NICID: # virtual NIC, starting from 1, corresponding to VirtualBox definition
  • PEER: Peer type, 1 for switches, 2 for hosts (e.g. servers, PCs)
  • DIFNO and DevType seem useless

The control message

Action=1 IFNO=1 NICID=2 PEER=1

means to bind the logical Ethernet port #1 (internally represented as something like GigabitEthernet1/0/1) of the device to virtual NIC #2 (the 2nd item in Network section, i.e. <Adapter slot="1" enabled="true" MACAddress="00E001021234"...), and detect the link status by sending and receiving a very strange PDU:

0000   47 46 45 4e 03 00 00 00 0b 00 00 00 00 00 00 00   GFEN............
0010   00 00 00 00 00 00                                 ......

If PEER=2, then the link status will be always UP.

What they adopted is an extremely unportable scheme, unless some network emulators support hooks on topology changes, it is impossible to conveniently have drop-in support for emulating H3C devices.

Running on QEMU and GNS3#

Get the disk images from HCL directory %programfiles(x86)%/HCL, where:

  • boot.vdi is located at resource
  • YOUR_DEVICE.vmdk is located at version, for example s6850_b70.vmdk is for H3C S6850
  • init.vdi is located at resource, as templates for data disks data0.vdi and data1.vdi

Then you can boot up the device using QEMU like:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

#!/bin/sh

sn=$(echo AQMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwMDAwQEBAQFBQ== | base64 -d)
qemu-system-i386 \
  -M pc-i440fx-3.0,acpi=off,x-south-bridge=piix4-isa \
  -accel kvm \
  -smp 2 \
  -cpu qemu32,+aes,+pdpe1gb,+x2apic,+pae,+nx \
  -m 512M \
  -hda boot.qcow2 \
  -hdb system.vmdk \
  -hdc data0.vdi \
  -hdd data1.vdi \
  -nographic \
  -serial mon:stdio \
  -serial unix:/tmp/h3c.sock,server,nowait \
  -smbios type=0,version=VirtualBox \
  -smbios type=2,product="id=1 type=2100 slot=0 dev=58 macAddr=32-0f-2e-a1-01-00",serial="$sn",version="mem=512" \
  -netdev socket,id=remote,udp=127.0.0.1:30000,localaddr=127.0.0.1:30001 \
  -device e1000,mac=08:00:27:21:ad:b4,netdev=remote \
  -net nic,model=e1000,macaddr=00:e0:01:02:12:34 \
  -net nic,model=e1000,macaddr=00:e0:01:03:12:34 \
  -net nic,model=e1000,macaddr=00:e0:01:04:12:34 \
  -net nic,model=e1000,macaddr=00:e0:01:05:12:34 \
  -net nic,model=e1000,macaddr=00:e0:01:06:12:34 \
  -net nic,model=e1000,macaddr=00:e0:01:07:12:34 \
  -net nic,model=e1000,macaddr=00:e0:01:08:12:34

To bring up an interface, construct the control packet like:

1

printf 'Action=0 IFNO=1 NICID=2 PEER=2 DIFNO=0 DevType=2\0\0\0\0\0\0\0\0\0\0\0\0' | nc -c -u 127.0.0.1 30001

To run in GNS3, a lot of details need to be filled in. This is the node JSON representation in the .gns3 file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99

{
    "compute_id": "local",
    "console": 5000,
    "console_auto_start": false,
    "console_type": "telnet",
    "custom_adapters": [
        {
            "adapter_number": 0,
            "adapter_type": "pcnet",
            "mac_address": "08:00:27:34:0d:69"
        },
        {
            "adapter_number": 1,
            "mac_address": "00:e0:01:02:12:34"
        },
        {
            "adapter_number": 2,
            "mac_address": "00:e0:01:03:12:34"
        },
        {
            "adapter_number": 3,
            "mac_address": "00:e0:01:04:12:34"
        },
        {
            "adapter_number": 4,
            "mac_address": "00:e0:01:05:12:34"
        },
        {
            "adapter_number": 5,
            "mac_address": "00:e0:01:06:12:34"
        },
        {
            "adapter_number": 6,
            "mac_address": "00:e0:01:07:12:34"
        },
        {
            "adapter_number": 7,
            "mac_address": "00:e0:01:08:12:34"
        }
    ],
    "first_port_name": "Control",
    "height": 48,
    "label": {
        "rotation": 0,
        "style": "font-family: TypeWriter;font-size: 10.0;font-weight: bold;fill: #000000;fill-opacity: 1.0;",
        "text": "H3CS6850-1",
        "x": -23,
        "y": -25
    },
    "locked": false,
    "name": "H3CS6850-1",
    "node_id": "5e948055-ee17-4ca5-89f6-b0d0be2cfe64",
    "node_type": "qemu",
    "port_name_format": "Ethernet{0}",
    "port_segment_size": 0,
    "properties": {
        "adapter_type": "e1000",
        "adapters": 8,
        "bios_image": "",
        "bios_image_md5sum": null,
        "boot_priority": "c",
        "cdrom_image": "",
        "cdrom_image_md5sum": null,
        "cpu_throttling": 0,
        "cpus": 2,
        "create_config_disk": false,
        "hda_disk_image": "s6850_boot.vdi",
        "hda_disk_image_md5sum": "ee0bbf78158292ecae65196b7c3cc59c",
        "hda_disk_interface": "ide",
        "hdb_disk_image": "s6850_b70.vmdk",
        "hdb_disk_image_md5sum": "afec8011ddc62cc0ede1bc13f83dff30",
        "hdb_disk_interface": "ide",
        "hdc_disk_image": "s6850_storage.vdi",
        "hdc_disk_image_md5sum": "90c4d3a0f8e5c45e7c1579ab71f51ce2",
        "hdc_disk_interface": "ide",
        "hdd_disk_image": "s6850_storage.vdi",
        "hdd_disk_image_md5sum": "90c4d3a0f8e5c45e7c1579ab71f51ce2",
        "hdd_disk_interface": "ide",
        "legacy_networking": false,
        "linked_clone": true,
        "mac_address": "00:E0:01:02:00:00",
        "on_close": "power_off",
        "options": "-serial unix:/tmp/%console-port%.sock,server,nowait -M pc-i440fx-3.0,acpi=off,x-south-bridge=piix4-isa -cpu qemu32,+aes,+pdpe1gb,+x2apic,+pae,+nx -smbios type=0,version=VirtualBox -smbios type=2,product=\"id=1 type=2100 slot=0 dev=58 macAddr=2a-b3-b4-9a-01-00\",serial=\"\u0001\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0003\u0004\u0004\u0004\u0004\u0005\u0005\",version=\"mem=512\"",
        "platform": "i386",
        "process_priority": "normal",
        "qemu_path": "/usr/bin/qemu-system-i386",
        "ram": 512,
        "replicate_network_connection_state": false,
        "tpm": false,
        "uefi": false,
        "usage": ""
    },
    "symbol": ":/symbols/classic/multilayer_switch.svg",
    "template_id": "f2c3af44-37f7-4381-a8ff-1e4eede19c88",
    "width": 51,
    "x": -153,
    "y": -30,
    "z": 1
}

Running in GNS3