Getting Paramiko To Work

I’ve had a lot of struggles getting Paramiko to work and today I’ve finally managed it.
Here’s my setup:

-bash-3.2$ cat /etc/redhat-release
 Red Hat Enterprise Linux Server release 7.1 (Maipo)

This is fairly important.

pip install paramiko

Didn’t work for me. Some Googling led me to believe I needed the python-dev package installed. So I tried:

yum install python-dev

This didn’t work, so I had to search for it. So I searched for it using:

yum search python-dev

The above is my new favourite command. It turned up:

$ yum search python-dev
 Loaded plugins: product-id, rhnplugin, subscription-manager
 This system is receiving updates from RHN Classic or Red Hat Satellite.
 ==================================================================================================== N/S matched: python-dev =====================================================================================================
 python-devel.x86_64 : The libraries and header files needed for Python development

I then did a:

pip install paramiko

And I was done!

BGP RIB Failure

An infrequent, yet interesting issue that comes up occasionally is when BGP encounters RIB failures. Usually, it takes the form of a prefix which you’d expect a router to learn via eBGP in its RIB being learnt via a routing protocol with a worse administrative distance.

To understand this problem, we first need to realise that “RIB failure” in a “show ip bgp” output implies that a route offered to the RIB by BGP has not been accepted. This is not a cause for concern if you have a static, or connected route to to that network on the router, but if you’re expecting it to be via eBGP then you can infer that something is misconfigured with your routing.

This can also be simplified to “BGP does not care about administrative distance when selecting a path”.

For reference, the path selection algorithm goes:

Network layer reachability information.

Weight (Cisco proprietary). Bigger is better.

Local preference

Locally originated route

AS path length

Origin code. IGP>EGP>Incomplete

Median Exit Discriminator. Lower is better.

Neighbour type. eBGP better than iBGP.

IGP metric to Next Hop. Lowest Router ID wins.

Verifying SSL Certificate Chains

Found this link very useful doing this:

http://www.herongyang.com/Cryptography/OpenSSL-Certificate-Path-Validation-Tests.html

Some useful commands:
Display a certificate:
openssl x509 -in test-cert-top.pem -noout -text

Display a certificate’s issuer:
openssl x509 -in test-cert-top.pem -noout -issuer

Display a certificate’s subject:
openssl x509 -in test-cert-top.pem -noout -subject

Verify a certificate:
openssl verify test-cert-top.pem

Verify a certificate chain with 3 certificates:
openssl verify -CAfile test-cert-bottom.pem -untrusted test-cert-middle.pem test-cert-top.pem
CAfile keyword indicates which certificate is used as the root certificate, with the -untrusted option being set to validate the intermediate certificate in the chain

Verify a certificate chain with 2 certificates:
openssl verify -CAfile test-cert-bottom.pem test-cert-middle.pem

A10 Health Monitors

This post is an equivalence check of A10 vs ACE probes/health monitors.

    ACE

ACE-A# show probe

probe : tcp-3121-probe-1
type : TCP
state : ACTIVE
----------------------------------------------
port : 3121 address : 0.0.0.0 addr type : -
interval : 10 pass intvl : 30 pass count : 2
fail count: 2 recv timeout: 5

--------------------- probe results --------------------
probe association probed-address probes failed passed health
------------------- ---------------+----------+----------+----------+-------
serverfarm : vip-11.95.79.90_3121
real : ip-11.95.79.68[3121]
11.95.79.68 1286028 1104 1284924 SUCCESS

interval – the time period health checks for a healthy server are sent
pass intvl – the time period health checks for a server marked “DOWN” are sent
pass count – the number of successful probes required to mark a server as “UP”
fail count – the number of unsuccessful probes required to mark a server as “DOWN”
recv timeout – timeout before a probe fails


a10-1[test-1]#show health monitor
Idle = Not used by any server In use = Used by server
Attrs = Attributes G = GSLB
Monitor Name Interval Retries Timeout Up-Retries Method Status Attrs
---------------------------------------------------------------------------------
tcp-443-monitor-1 30 2 5 2 TCP In use

Interval – the time period health checks for a healthy server are sent
Up-Retries – the number of successful probes required to mark a server as “UP” after it has come back fro a “DOWN” state.

The A10 has no concept of a “pass-interval” unlike the ACE. So the same time periods used for probes to servers in an “UP” state are used for those in a “DOWN” state. This is not going to matter 9 times out of 10, because the A10 is a lot beefier than an ACE. Probe overhead is not a lot. However, it does throw up errors if you’re using a conversion tool.

Checking Faulty Cables

I recently had to work with a 3rd part to diagnose a link between our devices and came across this handy command. The link in question was a pretty hefty (75m-ish) UTP cable run between a Cisco and HP switch. I have visibility of the Cisco switch, into the structured cabling into the patch panel, and the 3rd parties cable. Unfortunately I didn’t have a DC Operations tech with access to a Fluke, or the ability to interpret the output of a Fluke, but they did have a laptop with a 100Mbps NIC (this becomes important later on).

So I started by running the diagnostic on the production connection. It’s not working, so I don’t have to worry about taking stuff down. This gives me the following:

test cable-diagnostics tdr interface gi7/21
TDR test started on interface Gi7/21
A TDR test can take a few seconds to run on an interface
Use 'show cable-diagnostics tdr' to read the TDR results.

switchA#show cable-diagnostics tdr interface gi7/21

TDR test last run on: July 09 10:30:20
Interface Speed Pair Cable length Distance to fault Channel Pair status
——— —– —- ——————- ——————- ——- ————
Gi7/21 auto 1-2 77 +/- 6 m N/A Invalid Terminated
3-6 75 +/- 6 m N/A Invalid Terminated
4-5 75 +/- 6 m N/A Invalid Terminated
7-8 N/A 8 +/- 6 m Invalid Open

It doesn’t really tell me anything, apart from that there’s most likely, an infrastructure fault. This could be a faulty cable, patch panel, or a faulty switchport.

I now start my troubleshooting. I need to ascertain the path I have visibility of. The tech tells me there is structured cabling to the line card on the switch, so I get him to connect his laptop to the section the B end connects to, and I run the diagnostic.

switchA#show cable-diagnostics tdr interface gi7/21

TDR test last run on: July 09 10:31:55
Interface Speed Pair Cable length Distance to fault Channel Pair status
——— —– —- ——————- ——————- ——- ————
Gi7/21 100 1-2 16 +/- 6 m N/A Pair A Terminated
3-6 16 +/- 6 m N/A Pair B Terminated
4-5 N/A 12 +/- 6 m Invalid Short
7-8 N/A 11 +/- 6 m Invalid Short

Interestingly, this works, but at 100Mbps. I also see that a couple of pairs in the cable are not terminated correctly. Pair D is for 1Gbps and pair C is for PoE. This was turned up by a quick Google search. I also see that the fault is located +/-6m from the switchport, suggesting the problem is at my end.

I double check this against a working port:
Another_switch#show cable-diagnostics tdr interface gigabitEthernet 7/16

TDR test last run on: July 09 10:18:43
Interface Speed Pair Cable length Distance to fault Channel Pair status
--------- ----- ---- ------------------- ------------------- ------- ------------
Gi7/16 1000 1-2 0 +/- 6 m N/A Pair B Terminated
3-6 0 +/- 6 m N/A Pair A Terminated
4-5 309 +/- 6 m N/A Pair D Terminated
7-8 99 +/- 6 m N/A Pair C Terminated

I now reconnected the customer end, and set the port speed to 100Mbps manually using

interface Gi7/16
speed 100

Unsurprisingly, the port comes up. The cable diagnostic shows the same output as it did above. I suspect the problem is either a faulty punch down on pair D, an incorrectly crimped cable, or the customer manually setting their speed to 100Mbps for some reason.

Now usually, if this were structured cabling to a new switch, I would ask for the patch panel terminations to be punched down again. However, this time, I know the cable is the newest part in the puzzle and ask the tech to re-crimp it. It works, and is now up at 1Gbps.

I now have learnt something new about cabling pairs. There are a few things to be aware of.
– The “Distance to fault” field can throw up false positives.
– I do not know how a correctly terminated cable connected to a 100Mbps will output. Will it see the cable D termination even if it is not being used for communication?
– The connection drops momentarily when the diagnostic is run. Remember to be aware of this if troubleshooting a working connection.

Check 10Gb Interfaces On An ASA

I recently had to deploy and ASA pair. One of the pre-requisites is to make sure there’s an optic in the interface we’re going to use. On a switch you have the following options:

#show int te5/4 transceiver
Transceiver monitoring is disabled for all interfaces.

ITU Channel not available (Wavelength not available),
Transceiver is internally calibrated.
If device is externally calibrated, only calibrated values are printed.
++ : high alarm, + : high warning, - : low warning, -- : low alarm.
NA or N/A: not applicable, Tx: transmit, Rx: receive.
mA: milliamperes, dBm: decibels (milliwatts).

Optical Optical
Temperature Voltage Current Tx Power Rx Power
Port (Celsius) (Volts) (mA) (dBm) (dBm)
---------- ----------- ------- -------- -------- --------
Te5/4 27.0 0.00 7.6 -- -2.2 -2.7


Or

#show int tenGigabitEthernet 5/4 capabilities
TenGigabitEthernet5/4
Model: VS-S720-10G
Type: 10Gbase-SR
Speed: 10000
Duplex: full
Trunk encap. type: 802.1Q,ISL
Trunk mode: on,off,desirable,nonegotiate
Channel: yes
Broadcast suppression: percentage(0-100)
Flowcontrol: rx-(off,on),tx-(off,on)
Membership: static
Fast Start: yes
QOS scheduling: rx-(8q4t), tx-(1p7q4t)
QOS queueing mode: rx-(cos,dscp), tx-(cos,dscp)
CoS rewrite: yes
ToS rewrite: yes
Inline power: no
Inline power policing: no
SPAN: source/destination
UDLD yes
Link Debounce: yes
Link Debounce Time: yes
Ports-in-ASIC (Sub-port ASIC) : 1-5 (3-4)
Remote switch uplink: no
Dot1x: yes
Port-Security: yes

Or

#show inventory "Transceiver Te6/4"
NAME: "Transceiver Te6/4", DESCR: "X2 Transceiver 10Gbase-SR Te6/4"
PID: X2-10GB-SR , VID: V05 , SN: FNS14501PP3

Unfortunately, on an ASA, there is no way to check if a transceiver is present. You can use your CCO account to submit a feature request though.

Kill An SSH Connection

Check what’s connected to the switch first:

#show ssh
%No SSHv1 server connections running.
Connection Version Mode Encryption Hmac State Username
0 2.0 IN aes128-cbc hmac-md5 Session started user1
0 2.0 OUT aes128-cbc hmac-md5 Session started user1
1 2.0 IN aes128-cbc hmac-md5 Session started user1
1 2.0 OUT aes128-cbc hmac-md5 Session started user1

Kill session using “disconnect” command:

#disconnect ssh ?
The number of the active SSH connection
vty Virtual terminal

#disconnect ssh 0