# THE FAKE CISCO 

## Hunting for backdoors in Counterfeit Cisco devices

Dmitry Janushkevich

**F-Secure Consulting, Hardware Security Team**

Version 1.0, July 2020


-----

## 1 INTRODUCTION

Producing counterfeit products is, and always has been, a great business if you don't mind being on the wrong
side of the law. There’s no need to invest in a costly R&D process, and no need to select the best performing and
looking materials; the only criterion is the cost of manufacture. This is why we see many imitations of expensive
products on the market, and are likely to continue to see them being made and sold at a fraction of original’s
price.

Network hardware designed, manufactured, and sold under the Cisco brand is a perfect example of this. Having
an excellent reputation because of their great engineering, these products sell at a premium price point.
Naturally, this encourages some to try and produce counterfeits as it’s a way of making easy money. Stories of
such exploits abound in the media: a gang reportedly exporting[1] US$ 10 million worth of gear to the US, the FBI
seizing shipments[2] of fake hardware, and court rulings being issued[3] to stop the manufacturers. What does Cisco
do to combat fraud? Actually, a lot. Cisco has a dedicated Brand Protection organization whose purpose is to
defend against counterfeit and gray market activities. They partner with customs teams and regional
governments all over the world with success. In April 2019, they seized $626,880 worth of counterfeit Cisco
products in one day.[4] However, despite successful operations Cisco hasn’t been able to stop fraud fully. If there’s
an opportunity to make a fast buck, there’ll always be someone willing to take the risk.

In fall 2019, an IT company found some network switches failing after a software upgrade. The company would
find out later that they had inadvertently procured suspected counterfeit Cisco equipment. Counterfeit devices
quite often work smoothly for a long time, which makes it hard to detect them. In this particular case, the
hardware failure initiated a wider investigation to which the F-Secure Hardware Security team was called and
asked to analyze the suspected counterfeit Cisco Catalyst 2960-X series[5] switches. This initiated a research
project with the following goals:

- Verify no extra functionality such as "backdoor access" was introduced.

- Understand how and why counterfeit devices bypass the platforms authentication security control.

Naturally, as it’s not easy to tell genuine and counterfeit devices apart, to verify whether any kind of "backdoor"
functionality existed was also not easy, as it required a considerable amount of technical investigative work.
Ultimately, we concluded, with a reasonable level of confidence, that no backdoors had been introduced.
Furthermore, we identified the full exploit chain that allowed one of the forged products to function: a
previously undocumented vulnerability in a security component which allowed the device’s Secure Boot
restrictions to be bypassed.

This paper details the process which led to this conclusion and shares the technical knowledge gained during
this journey.

[1 https://www.pcworld.com/article/2920032/uk-gang-arrested-for-exporting-10-million-of-fake-cisco-gear-to-us.html](https://www.pcworld.com/article/2920032/uk-gang-arrested-for-exporting-10-million-of-fake-cisco-gear-to-us.html)

[2 https://www.infoworld.com/article/2653167/fbi-worried-as-dod-sold-counterfeit-cisco-gear.html](https://www.infoworld.com/article/2653167/fbi-worried-as-dod-sold-counterfeit-cisco-gear.html)

[3 https://www.sdxcentral.com/articles/news/cisco-wins-latest-battle-in-war-against-chinese-counterfeiters/2019/12/](https://www.sdxcentral.com/articles/news/cisco-wins-latest-battle-in-war-against-chinese-counterfeiters/2019/12/)

[4 https://blogs.cisco.com/partner/perform-transform-and-protect](https://blogs.cisco.com/partner/perform-transform-and-protect)

[5 https://www.cisco.com/c/en/us/support/switches/catalyst-2960-x-series-switches/series.html#~tab-documents](https://www.cisco.com/c/en/us/support/switches/catalyst-2960-x-series-switches/series.html#~tab-documents)


-----

While in this case no "backdoors" were identified, the fact the security functions were bypassed means the
security posture of the device was weakened. This could allow attackers who have already gained code
execution via a network-based attack, for example, an easier way to gain persistence, and therefore impact the
security of the whole organization.

### 1.1 Acknowledgements

This paper is the result of a huge team effort. The author would like to acknowledge Andrea Barisani’s
contribution, who was the first point of contact for the team and started the initial investigative work. Thanks
also go to Daniele Bianco and Andrej Rosano, who worked on the initial investigation. Furthermore, the author
would like to thank Thierry Decroix for numerous edits and reviewing this paper.

### 1.2 Disclaimer

As this work presents the results of practical research, some of the information that appears may be insufficiently
precise or incorrect. Please proceed at your own risk.

### 1.3 Device details

The following table details the devices the team had access to. The Genuine device was procured from an
authorized distributor and the manufacturer confirmed it was genuine.

Device type Name SW version

WS-2960X-48TS-L V05 Genuine c2960x-universalk9-mz.152-2.E7

WS-2960X-48TS-L V01 Counterfeit A c2960x-universalk9-mz.150-2.EX5
(as provided by the source)

WS-2960X-48TS-L V01 Counterfeit B c2960x-universalk9-mz.152-4.E7
(upgraded, resulting in breakage)

The devices will be referred to by their names where required.

|Device type|Name|SW version|
|---|---|---|
|WS-2960X-48TS-L V05|Genuine|c2960x-universalk9-mz.152-2.E7|
|WS-2960X-48TS-L V01|Counterfeit A|c2960x-universalk9-mz.150-2.EX5 (as provided by the source)|
|WS-2960X-48TS-L V01|Counterfeit B|c2960x-universalk9-mz.152-4.E7 (upgraded, resulting in breakage)|


-----

## CONTENTS

#### 1 INTRODUCTION ................................................................................. 1

1.1 Acknowledgements ................................................................................................................................... 2

1.2 Disclaimer .................................................................................................................................................. 2

1.3 Device details ............................................................................................................................................. 2

#### 2 ANALYSIS .......................................................................................... 4

2.1 Symptoms .................................................................................................................................................. 4

2.2 Exterior differences ................................................................................................................................... 4

2.3 Board analysis ............................................................................................................................................ 5

2.4 Boot log acquisition and analysis ............................................................................................................. 12

2.5 Content extraction from live systems ...................................................................................................... 13

2.6 Direct Flash content extraction ............................................................................................................... 15

2.7 Flash content analysis .............................................................................................................................. 16

2.8 Bootloader analysis .................................................................................................................................. 20

2.9 HBOOT patch analysis ............................................................................................................................. 22

2.10 SLIMpro analysis ...................................................................................................................................... 24

#### 3 CONCLUSIONS ................................................................................. 30

 4 ABOUT THE AUTHOR ....................................................................... 31

 5 ABOUT F-SECURE HARDWARE SECURITY TEAM ............................... 31

 6 APPENDICES .................................................................................... 32

6.1 The SoC .................................................................................................................................................... 32

6.2 The MZIP file format ................................................................................................................................. 35

6.3 The AMCC file format .............................................................................................................................. 36

6.4 Software signatures and keys ................................................................................................................... 36


#### INTRODUCTION ................................................................................. 1

Acknowledgements ................................................................................................................................... 2

Disclaimer .................................................................................................................................................. 2

Device details ............................................................................................................................................. 2

#### ANALYSIS .......................................................................................... 4

Symptoms .................................................................................................................................................. 4

Exterior differences ................................................................................................................................... 4

Board analysis ............................................................................................................................................ 5

Boot log acquisition and analysis ............................................................................................................. 12

Content extraction from live systems ...................................................................................................... 13

Direct Flash content extraction ............................................................................................................... 15

Flash content analysis .............................................................................................................................. 16

Bootloader analysis .................................................................................................................................. 20

HBOOT patch analysis ............................................................................................................................. 22

2.10 SLIMpro analysis ...................................................................................................................................... 24

#### CONCLUSIONS ................................................................................. 30

 ABOUT THE AUTHOR ....................................................................... 31

 ABOUT F-SECURE HARDWARE SECURITY TEAM ............................... 31

 APPENDICES .................................................................................... 32

The SoC .................................................................................................................................................... 32

The MZIP file format ................................................................................................................................. 35

The AMCC file format .............................................................................................................................. 36

Software signatures and keys ................................................................................................................... 36


-----

## 2 ANALYSIS

### 2.1 Symptoms

The biggest indication a 2960X device is a counterfeit is that it becomes inoperable after a software upgrade is
performed. This also happened to the victim company and the devices had to be replaced. During negotiating
the replacement with the vendor, the company found out they had unknowingly bought counterfeit devices.
Moreover, the CISO was brought in to initiate investigations as to whether the company’s networks had been
compromised.

While the device lost its primary function as a network switch when the software upgrade was installed, it could
still be accessed via the console. The following message was then displayed on the console during boot:
```
[Date Time]: %ILET-1-AUTHENTICATION_FAIL: This Switch may not have been manufactured by Cisco
or with Cisco's authorization. This product may contain software that was copied in violation
of Cisco's license terms. If your use of this product is the cause of a support issue, Cisco
may deny operation of the product, support under your warranty or under a Cisco technical
support program such as Smartnet. Please contact Cisco's Technical Assistance Center for more
information.

```
Reverting the software version did not fix the problem, likely pointing to evidence of data being overwritten
during the update process.

### 2.2 Exterior differences

Because clones and packaging are getting more realistic, many people don't realize they have counterfeit
network equipment until it's installed and begins acting strangely. However, it is possible to spot minor
differences in the visual appearance of the suspected counterfeits through comparison with a known-genuine
device. Presented below are the most prominent differences we found during our investigations.

**_Figure 1. The suspected counterfeit switch (on the left) has port numbers in bright white, while the known_**
_genuine device has them in grey. The text itself is misaligned. The triangles indicating different ports are_
_different shapes._


-----

**_Figure 2. The mode button’s shape is slightly different. The square next to the management port is greenish_**
_on the counterfeit switch. On the genuine device, it is bright yellow._

While immediately recognizing such minute differences may be challenging to spot, side-by-side comparison
clearly shows that the enclosures of counterfeit units are of a lesser quality.

### 2.3 Board analysis

The main component of any modern electronics is printed circuit boards (PCBs) carrying electronic components
such as integrated circuits, connectors, and passive components. By analyzing these boards, we could spot any
differences and similarities between devices of the same family in the hope of gaining insight into what
modifications were done by the counterfeiters.

The overall board layout of the three devices was similar, with the Genuine unit and Counterfeit B sharing more
similarities in appearance. When observed in detail, however, it was possible to verify modifications for forgery
purposes, and the differences are significant.

The absence of a holographic sticker on the counterfeit units was immediately noticeable. While its presence on
the Genuine unit was not a guarantee of authenticity, its absence typically indicated a counterfeit.

**_Figure 3. Legitimate holographic sticker which was absent on both counterfeit units._**


-----

**_Figure 4. Genuine unit, internal view._**


-----

**_Figure 5. Counterfeit A, internal view._**


-----

**_Figure 6. Counterfeit B, internal view._**


-----

The Flash part numbers were found to be different, albeit both identifying 1Gbit parallel NOR Flash devices. The
Genuine unit had Spansion p/n S29GL01GS11TFIV1 installed, while the counterfeit devices were installed with
Micron/Intel p/n JS28F00AM29EWH. This could be attributed to the fact that the Genuine unit was identified
as V5 while the counterfeits were V1. It is hard to say without comparing devices of the same version whether
this was an indicator. It could also be the result of the manufacturer swapping in cheaper parts.

**_Figure 7. U8, an 1Gbit NOR Flash. PCB rework evidence on the Counterfeit A unit: soldering flux residue on_**
_and around the Flash IC is present._

What was more concerning, was the presence of PCB rework traces around the Flash IC on the Counterfeit A
unit. While it could be the case the unit was legitimately repaired, no record of such activity was found via
available sources. This led us to conclude that, at some point, the Flash chip had been replaced. This may have
been part of legitimate repair activities; however, it is not typical for legitimate repair shops to leave flux residues
on the board.


-----

##### 2.3.1 COUNTERFEIT A

Compared to other units, the main board of the Counterfeit A unit had by far the biggest number of components
installed, as well as completely different Ethernet chips. Judging by that, we recognized the possibility of a V1
board, as suggested by the device type sticker located on the enclosure.

No irregularities were spotted during the inspection of the top-side components, but a prominent difference
was identified on the bottom side.

**_Figure 8. The Counterfeit A processor board had an "extra" component added._**

The component was connected through vias to the pins of U55 located on the top side of the board, identified
as a 512Kbit I[2]C EEPROM. While two wires connected to power rails, two connected to pins 5 and 6 of U55,
which carry the I[2]C SCL and SDA bus signals.

**_Figure 9. U55, a I2C EEPROM, ST p/n M24512 marked 412R8 as found in the Counterfeit A unit._**


-----

##### 2.3.2 COUNTERFEIT B

The Counterfeit B unit was found to have one significant difference when compared to other units: the presence
of components with their top marking removed with a laser; the components are U55 and U10006.

Whereas the same U55 component is a serial EEPROM in the TSSOP-8 package in other units, the Counterfeit B
unit sported a QFN16 package. This could be a legitimate engineering change when producing a new board
revision, but it is unusual for an I[2]C EEPROM to be manufactured in such a package. Furthermore, there is no
realistic reason to remove top marking for such a simple part.

**_Figure 10. U55, an unknown component with top marking removed._**

The other component with its top marking removed was U10006, bearing a nondescript top marking even in a
genuine unit: 1341604/QQ2Q8/B1837. Unfortunately, this made determining the exact function of this
component very challenging.

**_Figure 11. U10006, another component with top marking removed in Counterfeit B. The Genuine unit to the_**
_right for comparison. Note silkscreen quality was lacking on the counterfeit board._


-----

Note the many similarities in via positioning; they matched almost exactly and followed almost the same routing
for traces connecting to the passive components on the right. However, U10006 was not only rotated, it also
appeared to have a different pinout compared to the genuine board. Pin 1 marking was also very different both
on silkscreen and the component itself.

### 2.4 Boot log acquisition and analysis

Any sufficiently sophisticated device needs to have a way to provide an insight for manufacturing engineers and
end users into its state as well as enable controlling its functioning. Such a way is usually implemented through
some sort of a console, allowing the operator to observe system messages as the device boots and to input
commands to control the boot process and device operation in general. Naturally, these messages provide a
wealth of useful information when investigating device workings and as such are important to capture.

For example, this allowed identification of the software version installed on each device, and was instrumental
in obtaining clean images from the vendor for further comparison.

Inventory information was also displayed during the boot, mainly consisting of serial numbers for various parts
comprising the device. This allowed us to verify the numbers against labels present on each part. While not a
direct indication of being counterfeit, a mismatch was indicative of the part being replaced.

Probably the most interesting aspect was the analysis of the inoperable device, which failed the platform
authentication procedure. From the very start of the boot:
```
CPU rev: B
Image passed digital signature verification
Board rev: 18
Testing DataBus...
Testing AddressBus...
Testing Memory from 0x00000000 to 0x1fffffff.../
Using driver version 4 for media type 1 
...

```
It is worth noting that the device reported (some) digital signature verification passed, even though the boot
process resulted in a non-functional device. Similarly:
```
...
...done Initializing Flash.
Loading "flash:c2960x-universalk9-mz.152-4.E7.bin"...Verifying image flash:c2960xuniversalk9-mz.1524.E7.bin....................................................................................
............................................................................................
............................................................................................
............................................................................................
..............Image passed digital signature verification
@@@...@@@
File "flash:c2960x-universalk9-mz.152-4.E7.bin" uncompressed and installed, entry point:
0x3000
executing...

```
Note the two-pass loading process, with verification being performed separately.


-----

However:
```
...done Initializing flashfs.
Checking for Bootloader upgrade..
Boot Loader upgrade not needed(v)
FIPS: Flash Key Check : Begin
FIPS: Flash Key Check : End, Not Found, FIPS Mode Not Enabled
POST: MA BIST : Begin
POST: MA BIST : End, Status Passed
POST: TCAM BIST : Begin
POST: TCAM BIST : End, Status Passed
POST: ACT2 Authentication : Begin
POST: ACT2 Authentication : End, Status Failed
extracting front_end/front_end_ucode_info (43 bytes)
Waiting for Stack Master Election...
POST: Thermal, Fan Tests : Begin
POST: Thermal, Fan Tests : End, Status Passed
...

```
To summarize, the platform consisting of bootloaders, together with any potential pre-boot mechanisms
successfully authenticated the application image. However, the application then failed to authenticate the
platform. This seemed to correlate well with the high-level symptoms outlined before, which may have either
meant that the software image was patched covertly when loaded, or the patched version was already
provisioned; this would be easy to verify once images had been extracted from the devices.

### 2.5 Content extraction from live systems

The Cisco devices implement advanced management capabilities through the serial console, allowing not only
to change the configuration settings but also explore the available file systems. This was leveraged as a noninvasive method of extracting contents – we did not know in advance whether raw filesystem data extracted
from Flash ICs could actually be used with non-Cisco systems, for example, mounted in Linux.

This stage followed directly after boot log acquisition, and leveraged the console access to explore, and tried to
discover and back-up any interesting files present on the local filesystems. Unless a standard filesystem is used,
extracting files from a raw Flash image is usually significantly harder than copying those off the device when it is
powered on.

The devices store the application software image in a file located in the main flash: file system. Therefore, it
was relatively easy to obtain a copy of the software being executed on counterfeit devices. The fact that the
units stopped working after a software update, yet still reported the updated version during the boot, can be
considered evidence that no hidden software copy was present elsewhere on the system.


-----

```
Switch#
May 11 13:25:40.493: %USBFLASH-5-CHANGE: usbflash1 has been inserted!
Switch#dir flash:/
Directory of flash:/
  2 -rwx   1048576 Nov 26 2019 15:34:12 +00:00 test
  3 -rwx    4120  Oct 3 2019 13:37:32 +00:00 multiple-fs
  4 drwx     512  Jul 4 2014 02:51:46 +00:00 c2960x-universalk9-mz.150-2.EX5
 643 drwx     512  Jul 4 2014 02:51:47 +00:00 dc_profile_dir
122185728 bytes total (97654272 bytes free)
Switch#dir flash:/c2960x-universalk9-mz.150-2.EX5
Directory of flash:/c2960x-universalk9-mz.150-2.EX5/
  5 -rwx     800  Jul 4 2014 02:43:35 +00:00 info
  6 drwx    6144  Jul 4 2014 02:43:35 +00:00 html
 642 -rwx  18229248  Jul 4 2014 02:45:34 +00:00 c2960x-universalk9-mz.150-2.EX5.bin
122185728 bytes total (97654272 bytes free)
Switch#$copy flash: c2960x-universalk9-mz.150-2.EX5/mz.150-2.EX5/c2960x-universalk9-mz.1502.EX5.bin usbflash1:/
Destination filename [c2960x-universalk9-mz.150-2.EX5.bin]? CA_c2960x-universalk9-mz.1502.EX5.bin
Copy in
progress...CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC
CCCCCCCCCCCCCC
18229248 bytes copied in 15.103 secs (1206995 bytes/sec)
Switch#
May 11 13:27:00.622: %USBFLASH-5-CHANGE: usbflash1 has been removed!

```
The downloaded software image could then be verified from the unit by obtaining a legitimate copy of the same
software version from the manufacturer and comparing SHA256 hashes.
```
$ sha256sum CA_c2960x-universalk9-mz.150-2.EX5.bin 
1422575698b28bb6df41942e2147f696f0ce6be9f126cc4e308861047408a647 CA_c2960x-universalk9mz.150-2.EX5.bin

```
The hashes were found to match; no changes were made to the software image stored on the counterfeit unit.
This meant it was highly likely application software was patched in memory after load.

However, some information of potential interest is rarely stored in file systems. For example, boot loader
programs, or bootloaders, are rarely present there and are typically stored in "raw" form directly in Flash
memory. The only way to gain access to this code is through extraction of raw Flash memory content followed
by analysis of whatever content was actually extracted.


-----

### 2.6 Direct Flash content extraction

As already noted, the boards sported a prominent parallel NOR Flash chip of considerable size. This made the
chip the prime candidate for storing at least the application part of the overall software package, so was the first
one to be checked for suspicious traces.

All three devices underwent the same extraction procedure. First, the chips were removed from the boards.
Then, Flash content extraction was performed with the Elnec BeeProg2 with 70-3170 TSOP56 adapter, using
the Elnec software PG4UW. Content inspection showed the chips were written with bytes swapped with 16-bit
words, thus requiring a quick adjustment to accommodate for that.

**_Figure 12. Evident byte swapping in the extracted content. The text was intelligible but required a certain_**
_mental strain to understand it._

**_Figure 13. TSOP56 adapter used. (Image taken from the official Elnec web site)_**

Three images were obtained, each 128MB in size. After content extraction, the Flash chips were installed back
on the boards to return them to an operable state.


-----

### 2.7 Flash content analysis

The main purpose behind this step was to gather intelligence on how data was stored on the physical medium,
and whether there was anything not accounted for during the live system analysis step performed before, such
as bootloader code, any signatures, etc.

A quick content inspection using the entropy graphing feature of the binwalk tool showed us several areas of
interest.

- The high-entropy area of about 20M bytes was probably the main software image, fitting the size and the

entropy level as compressed data has high entropy

- What followed were two areas of distinct entropy footprints, likely some sort of uncompressed data

- The zero-entropy area is where the same 0xFF value was written and can be considered empty

- At the very end, there were several small but distinct blocks of data

After manual inspection at the very beginning of the image and direct binary comparison, we concluded that no
bootloader code of any kind was placed there; the data looked more like a file system. Supporting that
assumption was the difference in the composition of the blocks on the devices. Therefore, it made more sense
to assume bootloader code was located at the end of the Flash; this was supported by the similar-looking
entropy graph for all images towards the end.

**_Figure 14. Zoomed view showing the very end of the image. The picture was very similar for all three Flash_**
_images._

Indeed, inspecting the last megabyte revealed the presence of what appeared to be two bootloader programs,
easily identified with the help of embedded strings (examples from Counterfeit A):
```
C2960X Boot Loader (C2960X-BROM) Version 15.2(2r)E1, RELEASE SOFTWARE (fc1)
Compiled Wed 23-Apr-14 02:21 by abhakat

```
and
```
C2960X Boot Loader (C2960X-HBOOT-M) Version 15.2(2r)E1, RELEASE SOFTWARE (fc1)
Compiled Wed 23-Apr-14 02:21 by abhakat

```
These boot loaders corresponded to the last two peaks on the entropy graph above.


-----

**_Figure 15. Entropy graph of the known-good Flash image obtained from the Genuine unit._**

**_Figure 16. Entropy graph of the suspect Flash image obtained from the Counterfeit A unit._**


-----

The next series of peaks of around 130 to 132MB represent data that was hard to attribute to anything that was
known, but the best guess was that it was related to the Flash file systems. However, one interesting piece of
information at the very end stood out; it was marked with the magic string AMCC and contained what appeared
to be file names such as ppc.bin.key and pka_fw.bin. The analysis of this file format is summarised in appendix
6.3.

The peak around 127MB belonged to the microcode binary named `c2960x_dlpd_porter.bin. This file was`
located on the ucode0 filesystem.

Finally, two small peaks represented the lic0 and lic1 filesystems.

The following table summarizes the investigation of the last few megabytes of the Flash contents of the
Counterfeit A unit.

Offset Contents

0x76A0000 The lic1 filesystem

0x7720000 The lic0 filesystem

0x78C0000 The ucode0 filesystem

0x7DA0800 Block of data marked AMCC

0x7DC0000 Board configuration (text based)

0x7EE0000 C2960X Boot Loader (C2960X-HBOOT-M)

0x7FDD800 Inventory data

0x7FDDC00 Signature 1

0x7FDFFFC 4 bytes, 55 AA AA 55

0x7FE0000 C2960X Boot Loader (C2960X-BROM)

0x7FFDC00 Signature 2

0x7FFF000 Unknown; appears to be PowerPC code but no text strings to identify it

0x7FFFFFC 4 bytes, 4B FF F0 02 (decodes as the ba 0xfffff000 PowerPC instruction)

It can also be assumed the main `flash filesystem started right at offset zero. While offsets of the first three`
entries differed on other units, the overall composition was expected to be the same.

With some general understanding of what was where in the Flash, it was then possible to perform a meaningful
comparison of Flash contents between the genuine unit and the counterfeit one, which was not upgraded.

Looking at the very end of the file, some differences stood out.

Both counterfeit units had offsets 0x7FFF800..0x7FFFFFC and 0x7FFE000..0x7FFF000 filled with apparently
random noise. Given that Flash chips in the erased state had all bits set, it was unlikely this was uninitialized data.

|Counterfeit A unit.|Col2|
|---|---|
|Offset|Contents|
|0x76A0000|The lic1 filesystem|
|0x7720000|The lic0 filesystem|
|0x78C0000|The ucode0 filesystem|
|0x7DA0800|Block of data marked AMCC|
|0x7DC0000|Board configuration (text based)|
|0x7EE0000|C2960X Boot Loader (C2960X-HBOOT-M)|
|0x7FDD800|Inventory data|
|0x7FDDC00|Signature 1|
|0x7FDFFFC|4 bytes, 55 AA AA 55|
|0x7FE0000|C2960X Boot Loader (C2960X-BROM)|
|0x7FFDC00|Signature 2|
|0x7FFF000|Unknown; appears to be PowerPC code but no text strings to identify it|
|0x7FFFFFC|4 bytes, 4B FF F0 02 (decodes as the ba 0xfffff000 PowerPC instruction)|


-----

**_Figure 17. Differences at the very end between the Counterfeit A and Genuine units. While the last four bytes_**
_were the same, the counterfeit unit had some extra bits and pieces, unlike the erased genuine bits._

Unfortunately, further comparison was thwarted by the difference in versions of installed software and
bootloaders. It was necessary to locate intact copies of the bootloader code in order to perform any kind of
meaningful differential analysis.


-----

### 2.8 Bootloader analysis

Further analysis required obtaining software images of corresponding versions from the official sources.
However, first the software versions needed to be identified. This could be done via the file name of the installed
application image. Furthermore, the following bootloader image versions were present on the counterfeit units
at hand:

- Counterfeit A (operational):
```
C2960X Boot Loader (C2960X-HBOOT-M) Version 15.2(2r)E1, RELEASE SOFTWARE (fc1)
Compiled Wed 23-Apr-14 02:21 by abhakat
C2960X Boot Loader (C2960X-BROM) Version 15.2(2r)E1, RELEASE SOFTWARE (fc1)
Compiled Wed 23-Apr-14 02:21 by abhakat

```
- Counterfeit B (inoperable):
```
C2960X Boot Loader (C2960X-HBOOT-M) Version 15.2(4r)E3, RELEASE SOFTWARE (fc4)
Compiled Wed 04-Apr-18 10:35 by smaddasa
C2960X Boot Loader (C2960X-BROM) Version 15.2(2r)E2, RELEASE SOFTWARE (fc1)
Compiled Fri 05-Dec-14 01:35 by abhakat

```
The version strings appeared unique enough to allow a search for them within the uncompressed data sections
of the images at hand; see appendix 6.2 regarding the file format of the software image. In fact, they were
included as one monolithic image, likely directly copied to Flash in parts, starting from offset 0x7EE0000, taking
care to preserve inventory data. Bootloader images extracted from genuine software updates could then be
directly compared with their counterparts obtained from Flash content extracted directly from the units.

The `c2960x-universalk9-mz.150-2.EX5.bin software image obtained from Counterfeit A did not contain`
embedded bootloaders, so a search was conducted by grabbing multiple official images and matching version
numbers found in there. The c2960x-universalk9-mz.152-2.E.bin image provided the correct versions.

Checking the bootloader images obtained from Counterfeit A:

- Two changes were made to HBOOT: the first one to 8 bytes inside the code section, and the second which

was considerably larger, appearing appended to the HBOOT image

- Signature 1 data was intact

- 12 bytes modified right before the BROM code

- BROM code was intact

- Signature 2 data was different

- Extra data is present around the code block at 0x7FFF000, as highlighted before

At this point, the process in place for verification of each software piece was not known. However, it was evident
that some tampering with HBOOT had taken place. To understand these modifications and their significance
within the context of the system, it is important to consider what was known about the System-on-Chip and the
way it booted up. The collected reference data is summarized in appendix 6.1.


-----

Assuming the PowerPC core started executing at 0xFFFFFFFC, the Flash ROM could be safely placed at the upper
addresses so that the last 4 bytes in Flash, where we assumed the unconditional branch instruction was, map
starting at 0xFFFFFFFC.

After loading the last 1 megabyte or so of Flash into IDA Pro in the manner described above, we could start
exploring the code. The assumption made above regarding Flash mapping is validated by the fact that IDA Pro
could explore the code and create references automatically.

The branch instruction at 0xFFFFFFFC transferred control to the small blob of PowerPC instructions at
0xFFFFF000. This piece of code appeared to perform some basic initialization activities and passed control
further to BROM.

Judging by the contents of text strings included, BROM appeared to authenticate and start HBOOT and
provided a rudimentary set of commands to rescue the system from the state where HBOOT could be started.
While we were not as interested in the details of the command system, the image authentication functionality
was important to understand.

This functionality could be located easily by finding references to the tell-tale string Image passed digital
```
signature verification that was being printed on the serial console when image authentication succeeded.

```
The function referencing this string together with the companion failure message took 6 parameters, most
important of which were the base address of HBOOT and the signature 1 address. This meant signature 1 was in
fact the HBOOT signature.

By repeating the search and analysis on the HBOOT image, a similar function was identified that was used for
the same purpose of image authentication. This function was used to authenticate the application software
image; the process also showed up in console output. However, another use of the same function was made to
authenticate BROM code when it was being copied over to the Flash memory. Careful inspection of the
parameters being passed revealed that the BROM signature was located 0x2400 bytes before the image end,
which was exactly where signature 2 was located. We could therefore conclude that signature 2 authenticated
BROM code.

Further inspection of the signature verification implementation in both BROM and HBOOT showed the use of
some functions related to "SlimPro", as evident by corresponding error messages referenced by those functions:
```
WR: Timeout waiting for SlimPro response
RD: Timeout waiting for SLimPro msg
RD: Timeout waiting for SLimPro response

```
The answer to the question of what this component might be came from various materials published by the
vendor. The SLIMpro was a separate computing unit integrated into the System-on-Chip and was responsible
for system-related security operations. The strings presented above confirmed this information. Further
information on the SoC can be found in appendix 6.1.

With a clear overview of what was verified, when, and how, a prominent question arose: how did BROM report
successful verification of the modified HBOOT code? Answering this required a review of what was already
known, and a deeper dive into the modifications performed on the hardware of the Counterfeit A unit. But first,
the changes done to HBOOT and their purpose were investigated.


-----

### 2.9 HBOOT patch analysis

The analysis required a good understanding of what HBOOT did and how. We began by observing that like
BROM, HBOOT implemented a console with an impressive set of commands. Finding out how these commands
were added allowed us to spot every implemented command, and rename the corresponding handler functions.
This provided at least some insight into what parts were patched.

**_Figure 18. List of console commands supported by HBOOT._**

Starting with the first modification of two PowerPC instructions in the middle of the HBOOT code section, we
saw the boot command implementation was modified to include a call to the other added code fragment. The
call was patched in to be performed after the application image was loaded into memory and authenticated.

Analysis of the inserted code revealed this to be the first stage of a de-obfuscator (XOR-based with the key
derived from the unit serial number), processing the "random" data previously discovered at the end of Flash.
This data is de-obfuscated into a stack-based buffer, and control was then passed there. Care was taken to verify
the operation was performed correctly, so the unit did not crash even when there was no obfuscated data
present.

That code was found to be a stage 2 de-obfuscator, with the key based on certain data from the Flash IC not
accessible directly through conventional tools we had. Due to these circumstances, the key had to be brute
forced. Similarly, the de-obfuscated code of stage 3 was again placed into a stack-based buffer and executed.


-----

Stage 3 was found to be the actual patching code, searching for the `serialNu string and applying some`
modifications to the IOS image expected to be already loaded in memory. The modifications were few and
consisted of mainly "return OK" kind of patches; full details of functions being patched will not be published for
obvious reasons. However, it appeared the only purpose of this "added functionality" was to circumvent
software licensing protections.

It was due to this added patching functionality that the counterfeit units could bypass platform authenticity
verification. This also explained why units stopped working after a software update: the latest software will
almost certainly rewrite the patched HBOOT code, removing the work done to bypass the checks. The case of
the Counterfeit B unit confirmed this hypothesis.

When the CISO of the victim company provided us the counterfeit devices for investigation one of the main
tasks was to answer whether there were any backdoor-like functionalities being introduced. We concluded this
did not appear to be the case for application and HBOOT code.


-----

### 2.10 SLIMpro analysis

We had reached the point where we were ready to investigate the question posed previously with regards to
the bypassing of HBOOT image verification. As the SLIMpro component was responsible for authentication
because PowerPC cores only initiated the process by posting a message to a "mailbox", it was reasonable to
conclude that some changes were implemented, resulting in this component always reporting a successful
authentication. By identifying and understanding these changes we would be able to explain how the
counterfeiters are able to bypass the code authentication function.

What piqued our interest in the case of Counterfeit A was the "implant" added in conjunction with a serial
EEPROM chip. What was the reason for such an unusual and obvious addition? Furthermore, the same chip had
been replaced with a completely different package in Counterfeit B.

**_Figure 19. The implant PCB, disconnected from the unit, with the resin blob removed. As with Counterfeit B,_**
_the top marking was erased._

Since the protocol used to interface with EEPROM was relatively simple and slow, it was easy to intercept and
record the communications to gain insight into the workings of this implant. Any existing tool able to decode
I[2]C communications and export the decoded traffic could be used for this purpose. Below is a short excerpt of
this intercepted traffic as produced by the Logic software shipped with Saleae logic analyzer:
```
Time [s],Packet ID,Address,Data,Read/Write,ACK/NAK
3.256962416666667,0,'164' (0xA4),'0' (0x00),Write,ACK
3.257046583333334,0,'164' (0xA4),'0' (0x00),Write,ACK
3.257230166666667,1,'164' (0xA4),'240' (0xF0),Read,ACK
3.257315166666667,1,'164' (0xA4),'240' (0xF0),Read,ACK
3.257400333333333,1,'164' (0xA4),'3' (0x03),Read,ACK
3.257485416666666,1,'164' (0xA4),'18' (0x12),Read,ACK
3.257570500000000,1,'164' (0xA4),'0' (0x00),Read,ACK
3.257655583333333,1,'164' (0xA4),'252' (0xFC),Read,ACK
...

```
According to the M24512 data sheet, random-access read operation was performed by writing two address
bytes followed by reading multiple bytes of data. This corresponded to the observed traffic. A simple script was


-----

written to parse such output and to create a dump file containing the intercepted contents, as well as to provide
some overview on the read transactions executed.
```
Below is the output of the script showing what addresses were accessed and how many bytes read
(in hexadecimal), which is much easier to analyze compared to the raw transactions. It has
been abbreviated due to its size.Start address: 0000, byte count: 0020
Start address: 021E, byte count: 0004
Start address: 0020, byte count: 0040
Start address: 0060, byte count: 0040
Start address: 0060, byte count: 0040
Start address: 00A0, byte count: 0040
Start address: 00E0, byte count: 0040
Start address: 0120, byte count: 0040
Start address: 0160, byte count: 0040
Start address: 6320, byte count: 0400
[SEQUENTIAL ACCESS PATTERN CONTINUES]
Start address: 7320, byte count: 0400
Start address: 7720, byte count: 0364
Start address: 0060, byte count: 0040
Start address: 0060, byte count: 0040
Start address: 00A0, byte count: 0040
Start address: 00E0, byte count: 0040
Start address: 0060, byte count: 0040
Start address: 00A0, byte count: 0040
Start address: 00E0, byte count: 0040
Start address: 0120, byte count: 0040
Start address: 61A0, byte count: 0100
Start address: 6020, byte count: 0100
Start address: 0060, byte count: 0040
Start address: 00A0, byte count: 0040
Start address: 00E0, byte count: 0040
Start address: 6020, byte count: 0100
Start address: 0060, byte count: 0040
Start address: 00A0, byte count: 0040
Start address: 5EA0, byte count: 0100
Start address: 0220, byte count: 0200
[SEQUENTIAL ACCESS PATTERN CONTINUES]
Start address: 5E20, byte count: 0050
Start address: 0220, byte count: 0400
[SEQUENTIAL ACCESS PATTERN CONTINUES]
Start address: 5E20, byte count: 0050

```
Exactly the same behavior was observed when traffic was captured on the Genuine unit.

The following could be immediately noted:

- Repeating small accesses to addresses 0x60 through 0xE0 of 0x40 bytes each, followed by a significant

sequential read.

- Data starting at address 0x220 was read twice, but with different transaction sizes.


-----

Here is a closer look at what was being read:

**_Figure 20. The beginning of the obtained EEPROM/implant dump of the Counterfeit A unit._**

After a 32-byte block of unknown data, we could see the AMCC magic bytes, identifying the container format
described in appendix 6.3. By correlating the accesses with file offsets within the container, it was easy to identify
which files from this container were being read and in what order, so a higher-level overview could be pieced
together:

- `pka_fw.bin`

- `ipp.bin.ksg`

- `ipp.bin.key`

- `ipp.bin.sig`

- `ipp.bin`

- `ipp.bin (again)`

The fact that the `ipp.bin file was being read twice stands out. This mirrored the situation with the main`
application binary being read twice; first to verify the signature, then to decompress and pass control to.
Therefore, it is easy to assume the similar situation here as well: verify, then execute.

Such an implementation, however, is vulnerable to a classic race condition called time-of-check to time-of-use
(TOCTOU) where verified content could be manipulated after it had been verified but before its use. This
immediately prompted a comparison between the two reads (it was easy to truncate the I[2]C traffic dump to
exclude the second series of transactions starting at address 0x220).

Quite similarly to the HBOOT patch, two differences were detected: one small patch within the bulk of the file
and another, larger binary blob appended to the image.

Assuming this file contained software for some CPU architecture and not data, we could attempt to identify
which architecture this was meant to be executed on. Unfortunately, `binwalk -A did not produce any`
meaningful output. However, the following facts were observed:

- The beginning of the file appeared to contain a set of 32-bit little-endian integers similar in magnitude except

the very first one

- The byte sequence 70 47 was encountered quite often in the file

These two facts point at the possibility of this being ARM Thumb code, with the exception vectors located at the
beginning as is common with ARM-based embedded systems. From there, it was easy to verify the assumption
and guess the correct loading address with IDA Pro.

It was also possible to locate the previously observed BOOT FAIL string in this file.


-----

With that, we concluded that the ipp.bin file contained software running on the SLIMpro SoC component.

After loading the image in IDA Pro and spending some time marking up known library functions, we turned our
attention to the changes made to the image by the implant. It transpired that a call to memcmp() was replaced in
a certain function with a call to another function introduced by the patch; the new function inherited the original
semantics. Below is the pseudocode of the replacement function:
```
int __fastcall ADDED_sub_xxxx(const void *a1, const void *a2, unsigned int a3)
{
 signed int i; // r3
 int result; // r0
 // Check the conditions
 if ( (MEMORY[0x50000088] ^ MEMORY[0x5000008C]) == (MEMORY[0x2FFFFFF0] ^ MEMORY[0x2FFFFFF4])
  || *a2 == 0x27 && *(a2 + 1) == 0x4F )
 {
  n = 2;
 }
 // Compare the bytes
 for ( i = 0; i < n && *(a1 + i) == *(a2 + i); ++i )
  ;
 // Report the result
 if ( i == n )
  result = 0;
 else
  result = -1;
 return result;
}

```
Not having access to the SoC reference manual or other sources of information concerning the SoC memory
map as seen by SLIMpro, made it very challenging to understand what the first condition meant. The second
condition took into account the contents of one of the input buffers: it should have started with bytes 27 4F. In
both cases, the number of bytes to be compared was reset to two.

Considering the goal of the whole effort was to bypass signature verification checks, a reasonable guess was that
this function made that possible in some cases. On inspecting signatures present in the extracted Flash images,
we found the following BROM signature (after RSA decryption):

**_Figure 21. Decrypted BROM signature._**

An immediate conclusion was that the second condition was used to circumvent the BROM signature check,
while we could only assume that the first condition was somehow involved in circumventing HBOOT signature
check.


-----

The same exercise could be repeated against Counterfeit B. The integrated circuit installed on the processor
board appeared to have the same pinout as the one found on the implant PCB on Counterfeit A. This was
performed by tapping the correct vias on the board, given connecting to the pads of the QFN package footprint
was not feasible. Here is the list of transactions performed on the bus for this unit.
```
Start address: 0000, byte count: 0020
Start address: 0020, byte count: 0100
Start address: 0120, byte count: 0400
Start address: 0520, byte count: 0400
Start address: 0920, byte count: 0400
Start address: 0D20, byte count: 0400
Start address: 1120, byte count: 0400
Start address: 1520, byte count: 0400
Start address: 1920, byte count: 0400
Start address: 1D20, byte count: 0400
Start address: 2120, byte count: 0400
Start address: 2520, byte count: 0400
Start address: 2920, byte count: 0400
Start address: 2D20, byte count: 0400
Start address: 3120, byte count: 0400
Start address: 3520, byte count: 0400
Start address: 3920, byte count: 0400
Start address: 3D20, byte count: 0400
Start address: 4120, byte count: 0400
Start address: 4520, byte count: 0400
Start address: 4920, byte count: 0400
Start address: 4D20, byte count: 0400
Start address: 5120, byte count: 0400
Start address: 5520, byte count: 0400
Start address: 5920, byte count: 0400
Start address: 5D20, byte count: 0250

```
The differences were obvious: apart from two shorter reads at the beginning, the whole content was read in one
go, with no per-file read patterns as observed in Counterfeit A. Further inspection of the dumped data revealed
pseudo-random data with no discernible structure, apart from the first 0x120 bytes. The 32-byte header
contained the same data; however, the AMCC file structure was not found. Instead, it appeared the software
image starts as is. We could only conclude that the contents were encrypted in some way, and that no further
analysis was possible.

**_Figure 22. The beginning of the obtained EEPROM/implant dump of Counterfeit B._**


-----

Given that, there was an answer to the last question concerning the operation of Counterfeit A: a TOCTOU
vulnerability affecting SLIMpro ROM code was exploited in the wild to bypass software signature checks against
the SLIMpro secure processing unit. By extension, the issue affects the Genuine unit as well. While one previously
published report[6] regarding issues in the Cisco Catalyst secure boot process was accessible, at the time of
writing this paper, no public information was available detailing this or similar issues affecting the Catalyst 2960X series. This led us to believe this was indeed a previously unknown vulnerability.

It is important to note that the comparison of EEPROM data extracted from the Genuine unit and unpatched
data extracted from Counterfeit A showed them to be identical. As the patches were designed to bypass
signature checks only, we could conclude there was no "backdoor" code introduced into the SLIMpro
environment.

6 https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20190513-secureboot


Given that, there was an answer to the last question concerning the operation of Counterfeit A: a TOCTOU
vulnerability affecting SLIMpro ROM code was exploited in the wild to bypass software signature checks against
the SLIMpro secure processing unit. By extension, the issue affects the Genuine unit as well. While one previously
regarding issues in the Cisco Catalyst secure boot process was accessible, at the time of
writing this paper, no public information was available detailing this or similar issues affecting the Catalyst 2960-
X series. This led us to believe this was indeed a previously unknown vulnerability.

It is important to note that the comparison of EEPROM data extracted from the Genuine unit and unpatched
data extracted from Counterfeit A showed them to be identical. As the patches were designed to bypass
signature checks only, we could conclude there was no "backdoor" code introduced into the SLIMpro

https://tools.cisco.com/security/center/content/CiscoSecurityAdvisory/cisco-sa-20190513-secureboot


-----

## 3 CONCLUSIONS

The problem of counterfeiting is wide and raises a number of concerns. Not only does it mean a loss of trust in
the brand, and loss of revenue for the company whose products get copied, but counterfeit devices also pose a
security risk to the victim companies.

The two counterfeit devices provided to us for this research were detected after a software upgrade resulted in
a failure. These units were assessed from both software and hardware perspectives to investigate whether the
victim company’s networks had been compromised via introducing "backdoor access”, and to understand how
and why counterfeit devices bypassed the platform’s authentication security control.

Both units reached their goal of circumventing the implemented platform authentication checks with similar
means on the software level by relying on patching the loaded and authenticated application image before
control was passed over to the application. The functionality implementing the patches was wrapped into
multiple layers of obfuscation. However, these authentication bypasses were performed on each boot and thus
were not persistent.

No further functionality was identified on the software level that could be considered as backdoors – both in
PowerPC and SLIMpro code. This conclusion was supported by the fact that genuine software was patched onthe-fly and the patches only served to circumvent authenticity checks.

On the hardware level, the two units took quite different approaches as to circumventing boot-time software
authentication.

Counterfeit A contained "add-on" circuitry which exploited a race condition in the SLIMpro ROM code to bypass
SLIMpro software verification. It did this by intercepting EEPROM control signals, replacing certain bytes in the
image being loaded to modify software behavior. It appears the processor PCB in this unit was not modified.

While Counterfeit A only received a post-manufacturing add-on circuitry, the PCB design of Counterfeit B was
changed to incorporate the modification of Counterfeit A and replaced the EEPROM completely with an
unknown integrated circuit. This signified a considerable resource investment in design, manufacture, and
testing of such forged products compared to the more low-cost ad-hoc approach used in Counterfeit A. The
board layout and silkscreen similarities also suggested that the people behind this forgery might have either
had access to Cisco proprietary engineering documentation such as PCB design files in order to be able to
modify them, or they invested heavily in the complicated process of replicating the original board design files
based solely on genuine boards.


-----

## 4 ABOUT THE AUTHOR

Dmitry Janushkevich began his career as a testing- and later embedded-software engineer working on the
development of leading-edge solid-state drive technologies. Together with a bachelor's degree in computer
systems design, this has given him a strong background in embedded systems design and development for
future explorations in their security.

After joining F-Secure Consulting and gaining experience in customer-facing consultancy, embedded systems
security became his primary focus. Currently a senior consultant, he has a strong track record in providing
security-related consulting for automotive, aerospace, and consumer electronics industries.

## 5 ABOUT F-SECURE HARDWARE SECURITY TEAM

F-Secure Consulting’s Hardware Security team provides information security consulting to the most unique,
challenging and critical industries in the world. It delivers industry-leading services to secure hardware, safetycritical embedded systems, software applications and IT infrastructure.

It also provides detailed and comprehensible security analysis of software and hardware systems, along with
practical and effective mitigation and protection strategies.

With a vast breadth of experience in hardware and software design and engineering, it’s trusted by companies
across the globe to assess and test their products and processes. Its work safeguards products from malicious
compromise, and in doing so protects the safety of passengers, ensures the resilience of critical infrastructure,
and secures company trade secrets and intellectual property.


-----

## 6 APPENDICES

### 6.1 The SoC

This appendix details what little information was publicly available on the main System-on-Chip (SoC) in charge
of the device, paying special attention to any security features.

**_Figure 23. U1, the main SoC in situ in Counterfeit A. A couple of crystals and a backup Lithum battery can be_**
_seen around._

The main SoC was marked as APM86392-CNE600T[7] made by Applied Micro Corp (now MACOM). The
manufacturer describes the system as based around the Dual-Core Power™465 processor. No mention of
features related to code authentication during boot could be found on the manufacturer's website. However,
some details can be gleaned via announcements[8] in the media[9].

7 https://www.macom.com/products/product-detail/APM86392

8 https://linuxdevices.org/powerpc-soc-available-with-dual-cores/

9 https://www.embedded.com/applied-micro-adds-arm-core-in-cut-down-security-processor/


-----

**_Figure 24. Block diagram of APM86392. Image taken from linuxdevices.org._**

Any units relating to security were of particular interest; sometimes these units were not labelled as such but
were given cryptic or trademarked names. Two such units stick out: "Security Engine" and "SLIMpro". The
"Security Engine" was quite likely to be the one mentioned by the manufacturer as "security subsystem
(optional) with acceleration for IPSec, SSL/TLS, SRTP/SRTCP, Kasumi, and public-key protocols (PKA)" on the
product page. However, what is SLIMpro?

The same media source describing[10] a previous generation of the same SoC cites the manufacturer: "…
AppliedMicro also added its Scalable Lightweight Intelligent Management Processor (SlimPro) coprocessor,
which provides advanced power management, security, and concurrency features …" while also mentioning
Secure Boot and namedropping "SlimPro Trusted Management Module" which appeared to be relevant to this
research.

Searching for "AppliedMicro Trusted Management Module" yielded a very interesting – and apparently public
– presentation[11] documenting exactly that unit. We strongly recommended the reader to read through the
whole presentation, but here we note that the SLIMpro unit indeed represented the security epicentre of the
whole SoC.

10 https://linuxdevices.org/dual-core-15ghz-soc-touted-for-power-management-concurrency/

11 https://docplayer.net/995240-Appliedmicro-trusted-management-module.html


-----

Speculatively, and in accordance with what the presentation tried to show, the SLIMpro unit booted first and
was responsible for authenticating and starting any code on the PowerPC cores, apart from overall system
configuration tasks. This speculation was easy to verify by swapping NOR Flash chips between a counterfeit unit
and the genuine one. The genuine unit stopped booting completely, and the message displayed over the serial
console was:

```
BOOT FAIL

```

The counterfeit unit, on the other hand, proceeded to start the bootloaders as expected.

##### 6.1.1 THE BOOT PROCESS

There should be enough information – with some guesswork – to piece together the overall boot process from
the security standpoint, considering all known processing cores that participated.

1. SLIMpro start up

a) SLIMpro was expected to perform authentication of Flash contents
b) SLIMpro started (one?) PPC core, setting PC to 0xFFFFFFFC (most common configuration)

2. PPC core ran PBL code at Flash offset 0x7FFF000 (previously unidentified)
3. PPC core ran BROM code (by correlating console messages with BROM contents)

a) BROM authenticated HBOOT code

4. PPC core ran HBOOT code (by correlating console messages with HBOOT contents)

a) HBOOT authenticated application code

5. PPC core ran application code

a) Application code authenticated the platform
b) Depending on the result, the unit became inoperable


-----

### 6.2 The MZIP file format

Cisco delivers software updates for Catalyst devices as a single binary file. This meant updates for all system
components were carried in this file, whether application software, bootloaders, or microcode. This warranted
a closer look into the format of this file.

The file format was identified with the first four bytes being "MZIP" and was referred to as such. Apparently, this
was a "Cisco IOS MZIP compressed image" as noted on some sources on the Internet. No specifications were
publicly available.

Some existing tools[12] were found which served as a starting point for researching the format, however no tool
was found which would handle unpacking of the images at hand. Naturally, the code that was responsible for
loading MZIP files should serve as the best reference, so the already obtained genuine bootloader images could
be used for that purpose. Quite a lot of references to MZIP were found in the HBOOT image, so the relevant
parts were reverse engineered.

On a very high level, the file format was very similar to what would be found in executable file formats – this
could easily be seen from what mziptools was intended to output: some fixed header data including an entry
point address as well as a collection of segments. This meant the file was simply a program image which was
loaded and executed by the bootloader chain.

Segments could also be optionally compressed with PKZIP or BZIP2. Judging by the presence of the usual
```
BZh91AY... signature, data in our files was indeed compressed with BZIP2. As data is BZIP2 compressed, it was

```
possible to apply usual tools such as binwalk for data carving and decompression of relevant sections. While this
did not add much insight into the file format, it served as the first step to understand what was contained inside.

While provisions were present for more complicated arrangements, the contents proved to be very simple: one
code section and one data section.

Some trailer data was also included. This contained some textual information possibly related to the build
configuration and provided versioning information which we didn’t really need. Appended there, however, was
the image signature apparently used to authenticate the image, likely to be RSA2048 judging by the size. The
signature followed the same format used to authenticate both HBOOT and BROM images.

To facilitate future research, a dedicated tool was also developed to unpack and recreate MZIP files.

12 https://github.com/bvanheu/linux-cisco/tree/master/mziptools


-----

### 6.3 The AMCC file format

Data blobs marked with the AMCC magic bytes were found in at least two places:

- In the NOR Flash, U8

- In the dedicated EEPROM, U55

The AMCC (a reference to Applied Micro Corp?) file format was not overly complicated: a fixed-size header
followed by fixed-size file entries.

The header marked the format with the `AMCC magic bytes and then specified the entry count at offset 8. It`
included three data elements, one with no known meaning and the other two being offsets in the file pointing
at structures marked with AENV.

Each file entry contained a NUL-terminated file name of up to 16 bytes, with data offset at offset 16 and size at
offset 20.

The last 4 bytes of both the header and the file entries was likely to be a CRC32 value for integrity verification
purposes.

The illustration below highlights the main components of this file format:

**_Figure 25. AMCC file with meaningful data elements highlighted._**

While some unknown data elements remained, it was then easy to extract the files contained within.

### 6.4 Software signatures and keys

No public key data was directly identified in BROM or HBOOT, suggesting this data was loaded before control
was passed to BROM. So where was this key data stored? It was possible to make some educated guesses as to
where the public key(s) might be stored:

- In the ROM code of SLIMpro

- In some other location related to SLIMpro

- Somewhere on the NOR Flash

The easiest way to verify this was by browsing through the Flash images in the hope of stumbling upon the keys.
During Flash analysis we already identified one possible candidate: the AMCC data blob at Flash offset
0x7DA0800, with very conspicuous strings (file names?) like ppc.bin.key. This piece of data was the same in all
extracted Flash images, suggesting this data to be global for at least any device within the family – as expected
for software signing keys.


-----

Following the AMCC container format description (appendix 6.3), the files inside could be extracted.

This left us with the `ppc.bin.key file (the` `ppc1.bin.key is identical) which might have contained software`
verification keys, but its format was not immediately obvious. However, one could note that there were
repeating data pieces such as hex strings AB 12 34 CD and BE EF CA FE at the start and the end, and a text
string C2960X in the middle; all repeated 4 times. This could mean there were 4 keys contained within this file.

**_Figure 26. The beginning of ppc.bin.key and the BEEFCAFE marker._**

After spending several hours on decoding the format of these entries, we were able to provide the following
summary: the format loosely followed a tag-length-value (TLV) structure with 7-byte fixed header AE 02 25 AB

`12 34 CD` and 4-byte fixed trailer BE EF CA FE, tags were one byte, lengths were two bytes. Tag 04 was the RSA
modulus, and tag 05 is the RSA public exponent (always 0x10001, a typical value).

Two unique public key moduli were extracted from this file; they are reproduced below for reference. It is
important to highlight the fact these keys can only be used to verify signatures.
```
9f0269a7de698bfec8f57da81e51e9dd8c213134f17d07fcb83fb88d076afee659987d03d654319eb500e9902811
67d7595d36b25dba15a269d18a1fbd8fc5c6d1daffd93697d263aed7dc57c82266edbf0a35bd2896d6496819b60b
79ea16a0cd61fea77277361ee6bd4540cdd22149353d06659ff4ed03dad221d7b7fb7f0afcbe25c8a8dbda01270a
88d707aa0983c7ba4c1d1b218cc582e55ef7328ac52484ca3344193754d88b65ee1067f54cde2a46260be1b0e1f1
ddff96676f5e4dacd339f677d5d96c1a1a55a9e11e6930ed864576346c37e10f15f75fb8c3ecc2f6034ed2c6b002
de52ac78fa2e1a38acf80649fae7c193291c9b7f4018b5fc13ad
d12bad146859bb19cb3f0962f46edef40c3249373d8ae1dc243e735825b39073b2d0a507d658e4815eb0fceccce9
84741f2b69e8637388db967c337469001f1727201355242a60158fc5f84cce0bada77c626b33fc7334e1f8ebacfa
de0485721bdb6df4cf5c496deb69c152eb67fa2edf0bc8531a875fc6d7b9d29662d3022e894805956d76587f624b
0ed12cabbef4d91f4754bc1ae091070072610fb54d03d1efe7075b70e62473f914503186e550c2d5e4177cba2302
453cd8c07b78918604eb648a5dac02d4649a5d3bdeb6c1ac5a129e553f3905226d2abc291b0293cfe7d3b260bfb1
320c7186f6700ea4729b2f26a402cae22aedd2add7cc96ebf027

```
Investigating the signature format, we could conclude it followed the same TLV format albeit without headers
and trailers. Tag 0B was identified to be the RSA signature.


-----

Putting this knowledge to work, we could easily confirm whether our deductions were correct with Python:
```
>>> pubmod1 =
0x9f0269a7de698bfec8f57da81e51e9dd8c213134f17d07fcb83fb88d076afee659987d03d654319eb500e99028
1167d7595d36b25dba15a269d18a1fbd8fc5c6d1daffd93697d263aed7dc57c82266edbf0a35bd2896d6496819b6
0b79ea16a0cd61fea77277361ee6bd4540cdd22149353d06659ff4ed03dad221d7b7fb7f0afcbe25c8a8dbda0127
0a88d707aa0983c7ba4c1d1b218cc582e55ef7328ac52484ca3344193754d88b65ee1067f54cde2a46260be1b0e1
f1ddff96676f5e4dacd339f677d5d96c1a1a55a9e11e6930ed864576346c37e10f15f75fb8c3ecc2f6034ed2c6b0
02de52ac78fa2e1a38acf80649fae7c193291c9b7f4018b5fc13ad
>>> pubmod2 =
0xd12bad146859bb19cb3f0962f46edef40c3249373d8ae1dc243e735825b39073b2d0a507d658e4815eb0fceccc
e984741f2b69e8637388db967c337469001f1727201355242a60158fc5f84cce0bada77c626b33fc7334e1f8ebac
fade0485721bdb6df4cf5c496deb69c152eb67fa2edf0bc8531a875fc6d7b9d29662d3022e894805956d76587f62
4b0ed12cabbef4d91f4754bc1ae091070072610fb54d03d1efe7075b70e62473f914503186e550c2d5e4177cba23
02453cd8c07b78918604eb648a5dac02d4649a5d3bdeb6c1ac5a129e553f3905226d2abc291b0293cfe7d3b260bf
b1320c7186f6700ea4729b2f26a402cae22aedd2add7cc96ebf027
>>> pubexp = 0x10001
>>> sig =
0x8ec4a43c0658ea28ea529604116e56f2c5924cc865937ed4d33ad037eb95696dcce523d6f3f6f8d0724b48b6a0
1e1a6cd21984a9325ca72cf9a6c326502a565c0e7564ac7365f1e8e62d277f7072cdc22e861a17bfbce6d57fbfa7
ba2455b88389da2667e600b0599a0e069f7300c8ca14298c12db5440dab7007fc4e24369aff9313f1b98f35f0726
bc951f7076a12fdfa10447b93c824fae3aca57c20a6def3317a38a59936ea29a74a6d4e8696c7eb9aeab3b02f37b
6dae9c0ab9c01db40988581799f13838b3325b1f13ffbc012b89a6f2b8f2baec554578440f7e68b0c46ce525a487
b91f0f7fc628e7f952d8d3a8059b654d4c4b40b539b2b82e15cfab
>>> "%0512x" % pow(sig, pubexp, pubmod1)
'0001fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
ffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff
fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff003051300d0609608648016
503040203050004408ff1850fc5313c39ef350d8c2871797c8b42c5530ae6366bc47d3ea81d1ec8cceeb5d1087e5
6b1b330787445237a69b61730ff25d697cfe6e64ee3e77cad4489'

```
Judging by the well-formed decrypted data, we found the right software signature verification key; if the key
was incorrect, the decrypted data would appear random. The format appeared to fit what was defined in PKCS#1
v1.5[13] and the data told us the hash function used was SHA-512.

13 https://tools.ietf.org/html/rfc2313


-----