Testing Netmiko Connection to IOS Switch

I’ve played with NetMiko in the past, but never really documented anything I’ve done. It’s a cool tool, so I’ve set Remmina to scrape the SSH session and I’m planning to upload a few posts covering what I use it for.

This post today is just setting up Netmiko and validating function. Short and sweet.

First of all, you have to install NetMiko. I did this on Ubuntu with python3, so that meant dragging down python3-netmiko, python3-paramiko, and python3-scp. I used Synaptic Package Manage, but you can use apt-get or the package manager of your choice. Once you have NetMiko and it’s dependencies installed, pop terminal and open the python3 shell. My input is bold and notes are italicized below.

steve@ubuntu:~$ python3
Python 3.6.5 (default, Apr 1 2018, 05:46:30) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from netmiko import ConnectHandler

>>> cisco_test = {
... 'device_type': 'cisco_ios',
... 'ip': '192.168.0.1',
... 'username': 'cisco',
... 'password': 'password',
... }
>>> net_connect = ConnectHandler(**cisco_test) 
>>> net_connect.find_prompt()
'TestSW01#' confirms the connection to the switch
>>> output = net_connect.send_command("show ip int br") attempt to send a command
>>> print output
File "<stdin>", line 1
print output
^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(output)? Oops, forgot this is py3.
>>> print(output) 
Interface IP-Address OK? Method Status Protocol
Vlan1 unassigned YES NVRAM administratively down down 
FastEthernet0/1 unassigned YES unset administratively down down 
FastEthernet0/2 unassigned YES unset administratively down down 
...omitted for brevity...
GigabitEthernet0/3 unassigned YES unset administratively down down 
GigabitEthernet0/4 unassigned YES unset administratively down down 
>>>

Cool. It’s working! Now I’m going to jump in Atom and write some scripts for the next NetMiko post.

Testing ThousandEyes on Cloud VMs Pt.1

I played with Thousand Eyes back in 2015. I thought it was in interesting tool, but a tad on the buggy side. I don’t remember specifics, just that I had trouble getting ThousandEyes tests to run on the Google Cloud hosted Ubuntu boxes I spun up for that purpose. A colleague recently told me about his positive experience using this network insight tool, so I decided to revisit this. In this post, I’ll discuss deploying the ThousandEyes Enterprise Agent on two VMs: a clean Ubuntu 14.04 on Google Cloud and a WordPress serving Ubuntu 16.04 in AWS.

Note: If you’re following along, create an account for yourself on Thousand Eyes and make sure you can login to the agent dashboard.

Creating the Agents

This post isn’t about spinning up cloud VMs, but I’ll quickly go through what I did to create the servers I’ll be deploying the ThousandEyes agents on.

For the AWS WordPress build, I’m using Amazon Lightsail for simplicity. I navigate to my Lightsale instances and click Create Instance. From there I select “Linux/Unix” for the platform and, under Apps + OS, I select “WordPress.” I choose the “$5/mo with a month free” plan for testing, since I’m going to burn these VMs when I’m done, and click Create. 2 minutes of waiting for the machine to spin up and Voila! Out of the oven pops a WordPress server. I do go into the networking tab and assign a static IP as well.

Google Cloud is even simpler. From the Google Cloud Platform Compute Engine, I set this machine to 1vCPU with 3.75GB of memory and select Ubuntu 14.04 LTS as the boot disk. I leave everything else as defaults and hit Create. That was easy.

Deploying on the VMs

This process is nearly identical on both VMs, so I’m only going to go through this once. From the ThousandEyes Dashboard, navigate through Settings/Agents/Enterprise Agents and click +Add New Agent. Next to “Package Type” select Linux Package and you’ll see the following: Capture2.PNG

SSH in and follow the directions. Also, I kept the default log path, but you can change it if you want to. This the output you want, except for the few “[ OK ]” my snippet cut off because I resized my SSH clientThousand_Eyes_Deployed.

Now if you look at the bottom of your agent screen, you should see the hostname of the server you just installed Enterprise Agent on, like this:Added_VMs

Boom! You’re done. I do have 4 agents because I pre-built the top two in the image above to make sure I could do this before writing a blog post. Don’t worry about the N/A under utilization. It will go away in a few minutes. That’s it, though. Your agents are running and it was easy!

Setting up a test in ThousandEyes

This too is pretty easy, assuming you have at least a little networking experience. From the ThousandEyes Dashboard, navigate through Settings/Agents/Tests and click +Add New Test. I want to do a webpage load test to Amazon, so I’m going to go under “New Test” and select Web and Page Load. For the URL, I’ll enter the full protocol:string https://www.amazon.com set the interval for 2 minutes and select all four of my agents from the “Agents” dropdown. Note that you can run some tests from the ThousandEyes owned nodes from the dropdown as well. Go ahead and click Run Once at the bottom to verify function. Assuming good data comes back, click Create New Test.

Now navigate to Views/Tests and you got data.

data

The Takeaway

I’m impressed. The whole process took maybe 15 minutes for two servers. The instructions were simple and deployed the agent flawlessly. The test creation and associated output are both intuitive and useful. So, now I’m going to set up a few more tests and let these ThousandEyes collect data for a couple weeks. Expect part two of the series on ThousandEyes in early March where I’ll break down what else I’ve done and my impression over time.

Blogs and Podcasts for Networking Pros

As an aspiring networking pro, I am always looking for new ways to learn anything I can possibly cram into my brain. Unfortunately, I don’t have a whole lot of time to waste trudging through massive technical books. I already spend most of my home time cramming things like BGP, content switching policies, and data center facilities standards. Still, I want to learn more. Since I feel like vendor docs are full of bias, I prefer to use the filter of blogs and podcasts by people trusted within our industry to fill that gap.

Anyone who has been in networking for more than a few blinks has googled a problem while waiting for an answer from technical support. There’s a very good chance you found your answer on a blog. For example, I recently ran into a problem where I found an interface in an LACP Etherchannel that had dropped four times as much layer 2 receive traffic as the other interfaces in the port-channel. After consulting my Sr. Engineer, who pointed out the likelihood of a physical issue, I started searching for the problem. I was using a 7 meter passive TwinAx cable to go back to the core switch, which had been supplied by a vendor. It doesn’t get much simpler than passive TwinAx, it’s practically copper ethernet, so I started looking at the cable as the potential culprit. A little googling later and I found that passive TwinAx becomes unreliable at best beyond 5 meters. I swapped the run for OM3 on short-range SFPs and Viola! The interface counters normalized, even counting far lower than the other two TwinAx connected links. Thanks to a blog, I solved in an hour what would have been a multi-hour shuffle around by the vendor. I also took a moment to add the RSS feed of that blog to my outlook.

Podcasts solve another challenge: What should I do during my long boring commute? Well, I could move closer! Oh yeah, houses are expensive inside the perimeter. No thanks, I’ll just listen to podcasts while I drive. Podcasts actually pointed me down the path into infrastructure networking. I googled my way to the Packet Pushers Podcast as I was trying to sort out what I wanted to do in IT. Since then, I’ve enjoyed their take on the networking world 2-3 commutes a week (and I’ve joined Ethan Banks and Greg Ferro as a podcasts guest for two fun-filled shows). That leaves me with left over boring commute time, so I found more podcasts! Problem solved. I’m using my time to it’s fullest. If you’re new to podcasts, the process is simple. Search for the podcast by name in iTunes or your favorite podcatcher and add the podcast to your feed. Note: I prefer Stitcher. Not everything is on Stitcher, so I also have Podcast Addict on my phone for a few exceptions.

When I started to follow these blogs and podcasts I didn’t know the impact they would have on my professional work. I took my will to avoid breathy whitepapers (which still have their place) and boring commute and turned them into the opportunity to be more proficient in my job. I’m hoping these examples inspire you to add a few blogs to your RSS feed or follow a podcast or two.

This post is long enough, so I’m just going to list a few of the blogs and podcasts I frequent most often. If you have any blogs or podcasts to recommend, please do so in the comment section below. These are just my go-to resources, but I’d like to know what you use. Hope this helps you in your journey!

Podcasts

Blogs

Mission Accomplished: CCNA ICND2

7 months ago I wrote a post about the CCNA being impossible. I basically raged at Cisco for doing a terrible job at including everything needed to know to pass the test in their recommended study materials. I definitely still think that’s true, but I still managed to pass ICND2 yesterday! Now for the run down of the testing experience.

I’ll begin at the end. I left the testing center with an incredible mix of emotions. When I clicked “End Exam” I expected to get a $165 printout of the areas I needed to improve. That’s actually what I was going for. I didn’t expect a passing score. I just wanted to walk away with a plan to pass the next attempt. My chest wasn’t pounding with stress this time when I clicked the button. I was prepared to see a red score on the next page, so when I saw “Success! You scored 850 out of 811,” I honestly felt a lump well up in the back of my throat. After all the late night coffee and Monster fueled lab binges, after buying my 2nd copy of Odom’s ICND2 study guide just to re-highlight anything I didn’t know by heart, after earning honors by blowing away every exam in college and having my confidence stripped by 3 failed ICND2 attempts…there it was when I least expected it.

And I really didn’t expect it. This attempt was by far the worst Cisco exam I have taken. The wording was not just obscure, it was downright cryptic. I blew through the labs, but spent multiple minutes on some multiple choice questions because I couldn’t figure out what Cisco was asking. It was like the test was written in another language and run through Google Translate. Worse yet, I believe multiple questions exhibited poor technical word choice. It would be a question such as “Spanning tree interface behavior x [left out so cisco doesn’t sue me] is performed in which of the following modes.” It would then list out all of the interface roles and status, with both a role and status matching. Now pick one. Only one. As in radio buttons, not boxes. So you’re to ascertain from the question whether it’s a role or a status, right? Wrong! There no such thing as an interface MODE in spanning tree. This isn’t trunking!

The good news is I finally didn’t see any frame relay or RIP on the test. The IP addressing in questions was also about 50% IPv4 50% IPv6, so Cisco is really getting serious about IPv6. That I definitely loved.

I must not be too disgruntled. After I finish writing this sentence, I start studying CCNA Security.

 

Materials used include:

Books

Labs/Sims

Video

  • ITProTV ICND2
  • CBT Nuggets ICND2
  • LiveLessons ICND2 via SafaraBooksOnline.com

Net Neutrality: A Simple Perspective

I wasn’t around for Arpanet, the birth of Cisco, or the first wireless deployments. I’ve only been working with network technology for a handful of years. Still, in my naivety, I think I have a pretty good grasp of what people want their internet to look like. We don’t need the opinions of news analysts, politicians, or C level executives to break it down for us; it’s fundamental to who we are. We want freedom and we want privacy. Right now I already don’t have the freedom to choose my ISP, I can only choose DSL or cable broadband. In my case, AT&T or Comcast.
I have AT&T now, and I do like them, but I had Comcast broadband for years. I hated Comcast’s poor connection quality and snarky customer service. My only other option was lower bandwidth from AT&T DSL. I didn’t want less bandwidth, so I kept Comcast’s terrible service. So, why not just switch to another cable provider? Well, you can’t. In the United States, the cable companies spent the cable TV’s infancy lobbying for legislation that would allow each company to stake their claim in an area. Once cables were buried or strung, a cable provider essentially owned the ground and poles and could deny another cable provider a path to hang their lines. In some areas, this is permanent turf while in others it may only guarantee exclusivity for a period; ten years in my area. Fast forward to 2017 and most of these laws are still in place, so to this day if you don’t want Comcast, you can’t go talk to Charter for better service.
So we’ve established that you can’t shop around. Now let’s add net neutrality. Net Neutrality was intended to protect consumers from ISP abuse by forcing ISPs to treat all network traffic equally. It basically says if I cannot throttle a traffic for a service, for example, Hulu just because I own say Netflix and want my users to buy Netflix. Since I don’t own Hulu, you will have to buy Hulu bandwidth from me on top of your normal subscription to Hulu. This way, I get paid whether you pick my Netflix or the other guy’s Hulu. I am grossly oversimplifying, but in this capacity, it is similar to stock market protections against insider trading.
I don’t really have the in-depth insight into the matter, so if you know more about why we should preserve Net Neutrality CLICK HERE. I said this would be a simple perspective, so here it is: Killing Net Neutrality is wrong. You already don’t have the freedom to choose providers and, if we don’t preserve Net Neutrality, the greedy suits at the top of media get to pocket more greasy dollars at your expense.

Call your congressmen and tell them to preserve Net Neutrality.

NetScaler 11.1 to 12 Migration Review

I recently had the opportunity to upgrade Citrix Netscaler from v11.1 to v12 for a client. It was a relatively simple load-balancer on a stick architecture with high availability (HA) active/passive pair, so seems super easy, right? It was…mostly. I had two bumps along the way, so I wanted to put this out there. Oh yeah, this is also an appliance pair, but I’m withholding the model. Here are the Citrix recommendations for the process below.

I’m a new kid on the networking block, so I wanted to do this via the web GUI. This couldn’t be much simpler. After logging in on a NetScaler, click Configuration on the bar, which would put you in System. Herein lies the System Upgrade button; however, we’re not ready for that. This is an HA pair and we want to control our fail-overs.

Prep for Upgrading the Secondary Device

So you should just be able to upgrade your secondary device, right? Yes. The primary will see the secondary down and just keep on trucking…assuming you don’t run into a bug. Well, my boss will light me up if I trip over a bug and take down production, so let’s control this process.

First, we log into our primary box, navigate into System/High Availability, select the primary load-balancer, and click edit. This is also a good time to save any changes to the running-config, which will be noted as a orange dot on a blue file (seen in the top right of the snippet below. Mine’s grayed since there are no pending changes).

HA_Primary_1

In Configure HA Node, pull down the High Availability Status dropdown, select “Stay Primary” and hit the Okay button at the bottom of the form.

HA_Primary_1_2

In the High Availability page, the node state should now say STAYPRIMARY.

Now I log in to the secondary NetScaler and repeat this process, but this time I’m putting the secondary box into “STAYSECONDARY.” If you have HA Synchronization and HA Propagation checked checked as I do in the screenshot above, you can technically configure the secondary into stay secondary from the primary, but I don’t. I don’t like to wait for the config to propagate and I need to verify on the secondary that it’s correct anyways.

Once Primary is in stay primary and secondary is in stay secondary, it’s time to upgrade.

Upgrade Secondary Device

From the GUI on the secondary node, open the main System page and click System Upgrade.

System.PNG

Seen below, the GUI allows you to select the build from either your local machine or the appliance. Last time I updated one of these, the local file upload did not work, and would just spin after the upload  I tried it anyway, which failed miserably on IE, Chrome, and FireFox. I opened FileZilla client and transferred the build file to /var/nsinstall. Now I can select the file from appliance in the Select Firmware drop-down. Put a check-mark in Reboot after successful installation and click upgrade. A black progress box will pop up. In my experience, it’s not terribly trustworthy. Both of the ones I upgraded took almost exactly seven minutes each from clicking upgrade to logging back into the GUI. Go get some food and hit the bathroom. Maybe not in that order.

SystemUpgrade

Once the GUI comes back up, it should look slick indicating a successful upgrade. Need more proof? I do, but I somehow can’t find the build in the GUI since version 11, which Citrix swears is at the top of the screen. I SSH into the box and run > show version. It should reply with something like NetScaler NS12.0: Build redacted.nc, Date: Sep 22 2017, 09:11:54. I verify the back-end applications are functioning and your active users are happy. We’ve won half the battle. Now its time to break the network.

Prep for Upgrading the Primary Device

This shouldn’t break the network, but any sessions will need to renegotiate. If you’ve been following along and haven’t told your change board (shame on you), go tell someone because it’s fail-over time.

Log into the primary and secondary nodes, go back into System/High Availability, and set them both to ENABLED, changing the primary first. Let it bake for five minutes. Now since we don’t want to send commands from v12 to v11 (because that would be begging to hit a bug), from the v11 unupgraded primary node put a check in the primary’s checkbox, click the Action drop-down, and select Force Failover. There may be a pop-up or there may not. I don’t remember and this screenshot is in production so I’m not really gonna do it.Fail_Over_1

Confirm on the upgraded node that it’s now showing primary and the v11 is showing secondary. If that’s fine, go back into System/High Availability on both nodes and set the v12 node to STAYPRIMARY and v11 node to STAYSECONDARY. Verify this change and we’re ready to upgrade the remaining device.

Upgrading the Remaining v11 Device

I’m not going to rewrite this part, so in short: transfer the build file to the v11 node, upgrade it, and make sure it upgraded successfully. Now go back into System/High Availability on both devices and put them both back into ENABLED. I like to force one more fail-over to make sure both devices both handle traffic well. That’s it. As for the other bump I mentioned, it was my fault and I don’t want to talk about it. Hope this guide can help some of y’all out.

CCNA and Network+ Study Plan

I’m now building a page of networking study plans. My intent isn’t to teach here on the site, but to show what books, video series, sites, and labs that either I’m using for my own studies or that people I trust recommend. If you think I’m missing any great resources for Net+ or CCNA studies, please comment your recommendation.

Parallel to the study plans I’m also building a YouTube series called Network Speed Guides. These videos will address the topics of Net+ and CCNA R&S in the most condensed way possible. I’m aiming for a 5-10 minute video per topic designed to run through the terms, values, algorithms, etc. that have a bad tendency of falling out of your head.

These are both lengthy undertakings, so I’m giving myself 6 months to complete both. I appreciate you patience and look forward to your feedback!

CCNA R&S is Impossible 

Well, it has been for me.

I failed Cisco’s ICND2. Actually, I’ve now failed it for the fourth time, sort of: twice on version 2 and now twice on version 3. I am going to retake it. Each failure is demoralizing, but I’ll take it again anyways.

I have to admit I’m getting pretty sick of having my teeth kicked in by Cisco. While there’s a little bit of pity party sprinkled throughout this post, I’m also legitimately quite irritated with the test writers.

I feel like my last failure (my first attempt at ICND2v3) was an honest loss. The revision was only a couple months old. I could tell Cisco hadn’t yet made the questions terribly complex or obscure. I hear Cisco adds fluff to the questions to keep cheaters from memorize them for test banks, kind of like salting passwords in a database. I could see the fluff in the old test, but the revision was concise and well thought out. The questions we’re tough, but felt fair. That time I left the test center feeling like I failed a test because I didn’t know the material to the level they were asking of me. That’s the point of the exam, right?

That is not the case at all this time. I feel like Cisco sold me a lemon. The questions were once again obscure and cerebral. Worse, five or six of the questions literally left me wondering what they were even asking. Not like “Ah, they are trying to see if I understand the difference between STP State and Role! Cisco, that old fox,” but instead “Is this grammatically correct, or do I need to go back to 2nd grade?”

Making matters worse, Frame relay is back. Not all of it, just light theory and topology stuff, but I don’t know frame relay. I learned it for the previous version of the test, but I poured that right out my head after the revision. Frame relay has been a war-story of bygone days longer than my IT career, but I guess I better read that frame relay section in the appendix of the cert library.

IPv6 is much heavier this time. Now, that doesn’t bother me. I love IPv6. It feels so much more intuitive and I even lab more in v6 than v4. While I know the v6 threw me some gimme questions, only a minority of my real world networking has been v6. I don’t need to use math to subnet because I’ve worked with IPv4 almost every day for the last 4 years; I have the masks, CIDRs, wildcards, and binary in my head. I love IPv6, but I don’t have that level of confidence with v6 yet.

What really upsets me most is that I feel more than ready. I’ve read the Lammle and Odom big books cover to cover. I read a chapter from “31 Days Until Your Routing & Switching Exam” every night, then I reread it the next day on lunch. I’ve watched the entire CBT Nuggets twice, the O’Reilly series once, all of the condensed and full ITProTV videos, and even the LiveLessons in my Safari Books Online. I’ve done the Cisco, Transcender, and Boson practice tests, including rocking a Boson I’ve never seen before with a 993/850 two weeks ago; so close to perfect. Oh yeah, I also do this every day at work. Somehow I still scored a 766/811. I don’t know what’s left to do.

So, I’m going to take it again in June. I’m going to keep hammering my labs, building my Quizlet (which you may use), and taking the practice tests that I’m now memorizing the answers to. I just got back from Barnes and Noble with the hardback cert library for the revision (I already have v3 in electronic and v2 hardback). I’m going to read every single word in that book again, maybe a little faster this time, and compare it to my notes. I’m also going to write every command I come across and make sure I use it in my lab 5 times for each command, switch, and variable.

I’m so sick of taking this test. My employee evaluation, the way my coworkers see me, even my self-assessment of me as a young network engineer and aspiring infrastructure architect take a hit each time I see that sub-par score on the Pearson screen.

That’s why I’m taking it again. I’m better than an exam. I’m going to beat Cisco.

Don’t Believe the Programming Hype

I had the privilege of joining the Packet Pushers again recently to discuss the hype-train surrounding the alleged future death of the Network Engineer and rise of the omnipotent Network Programmer. We recorded this a few weeks back and I’ve been pondering on what to add to the conversation ever since. I’ve now decided that I have nothing to add. This was a great discussion. Have a listen!

http://packetpushers.net/podcast/podcasts/show-332-dont-believe-programming-hype/

A Very Tough Decision

A few months ago I had a very tough choice to make. I was happy with my job at Apogee Telecom. Great employer, good benefits, awesome environment, and a dependable partner at my site. Work was fun. I had been getting recruiter calls, but I had been with Apogee for about 3 months and I couldn’t see myself going anywhere else for quite awhile. Apogee is a great company. As a matter of fact go check them out at http://www.apogee.us. They have constant job postings and you will love them.

One normal day I picked up a call from Kristin Miller at Corus 360. I normally would let recruiting calls go to voicemail, but I like the folks at Corus 360, so I picked up the call thinking it would be just a casual “no thank you, work is great. How are things on your side?” kind of chat. That was the plan. Hey, Steve. how’s it going. Good, how about yourself? Oh, I’m doing well. Hey, I know you aren’t really looking, but I have a position for net engineer at Dingleboppits Hospital I thought I could tell you about. No, I’m happy where I…wait…did you say Dingleboppits? Yep, that’s the one. Long story short I told her she could send my resume over. And yes, *spoiler* Dingleboppits is a metaphor for NAME WITHHELD FOR SECURITY even though it wouldn’t be hard to figure out who I work for.

Sidestory, I was laid off in summer of 2016 in a massive workforce cut. The very first place I applied to was Dingleboppits Hospital. Some people have Google, Microsoft, the NSA; they can have them. I wanted to work for Dingleboppits. For a couple years I had been struggling with whether I wanted to go into big data, or ISP, or healthcare, but I had firmed up towards the latter over time. So then why did I want to be at that specific hospital? Well, my daughter was born at Dingleboppits, and both of my grandparents and even my wife had surgery there. During all of that stuff every nurse, doctor, tech, and maintenance worker I saw seemed to have a smile on their face. Further, I had worked, outside of IT, for one of their competitors during college. During that time I watched nurse after nurse leave for the promised land that is Dingleboppits. Part personal pride and part grass-is-greener, but that’s why I wanted to be part of their team. Unfortunately, this time around, I didn’t hear back from them, so a couple months later I was working for Apogee.

The next time around, when Kristin reached out to them, they must have liked what they saw, because two months later I stepped into my new cube at Dingleboppits.

But the decision to stay or go was hard; one of the most stressful choices I have ever made. I was already happy with my job and I really hate leaving an employer right after they spun down their employment search. So did my best to weight the options. Apogee: great company, good team, enjoyed my work, and room to grow. Dingleboppits: great company, really enjoyed meeting the team, should enjoy my work, and room to grow. What is the right choice? Well, I can do this here, but the other lets me do this and they both do that but this one does that better although in a year the other may do that better. I was so stressed about it that I got sick and irritable for about a week. Then I stopped for split second to breathe and I realized that I was trying to be unemotional about a choice that would affect the course of my life. I was using near mathematical methods to make a decision that would affect the fulfillment I would feel for myself; that can’t be the right way. One moment my head was spinning and the next I was calm, collected, and the decision had already been made up for me. I was leaving. Not because of anything Apogee had done to make me want to leave, but because I simply wanted to be where I wanted to be.

I wasn’t unhappy before, but I am happier now. If you are struggling with a decision like my own, or just a decision in general, don’t forget to stop and breathe.

If you are just looking for work in IT in Atlanta or North GA, definitely reach out to Kristin Miller or anyone else at Corus 360.

If you are looking for work in networking, definitely reach out to Apogee Telecom.