Log in

No account? Create an account

Selecting Best Database Engine for Netflow Storage

Well, if you are reading this article you probably facing pretty tough question of selecting one and likely already tried

some of the existing database engines like Microsoft SQL Server, MongoDB, PostegreSQL, MySQL or even Oracle and not happy with results.

You already turned off all the indexing, implemented daily partitions,  decided not to store some less important NetFlow fields  and may be even

upgraded your storage to those fancy M2 SSD and still your NetFlow reports takes minutes to appear.

If your NetFlow rates are somewhere under 1,000 flows per second you can skip this reading, pick any  DB and it will produce acceptable results.

Problems becomes visible when your flow rates  reaches 2,000 fps and at 10,000 fps just doing INSERT to your tables takes 70% of the time.

10,000 flows per second produces 600,000 database records every minute and only INSERT statement takes around 40 second to process leaving only 20 seconds of

available time  for any reports to be generated. The main problem here is that conventional DB engines are not optimized for storing read-only sequential data such

as time based event logging or NetFlow. Best database engine for Netflow has be designed with read-only sequential access in mind, no "Delete-Update" functionality is required for NetFlow.

This restriction allows great simplification of DB internal formatting structure  and processing logic. Second DB feature that best suited for NetFlow is reduction of possible Indexes to one.

Read more...Collapse )

Proactive vs Reactive network monitoring with Nectus

Most of the enterprise grade monitoring tools allow predefined thresholds to be set for specific monitoring parameters such as interface utilization,

RAM utilization or percentage of free TCAM available on switches. When a specific metric exceeds a predefined threshold you will receive an email alert, text message, or...

a phone call from your manager.  Likely those thresholds are tuned to a higher side to prevent false positives or filter out the events with a short duration.

But in any case alerts will be sent after critical conditions have already occurred and damage has already been done. This approach is called reactive network monitoring

and it has its value. But what if you can get an alert before critical conditions actually happen?  Nectus NMS is a pioneer of next-gen proactive monitoring and in selected

case-studies it demonstrated an ability to forecast critical operational conditions up to 1 week in advance with 80% accuracy.  Nectus analyses daily, weekly  and monthly fluctuations of network

operational metrics such as traffic volumes, interface utilization and extrapolates it based on polynomial curves for next 7 days.

This advanced mathematical approach demonstrated 80% accuracy in prediction of critical operational conditions for next 7 days and 95% accuracy for next 30 days.

The more time you have before operational conditions actually reach critical thresholds, the more options you  have to fix it before it impacts production.

Read more...Collapse )

Next Generation of Network Discovery and Monitoring Tools

There are many reasons why your network needs to be monitored. It is essential to any network administrator to keep track of the network’s performance and usage in real time and to detect failures, slowness or any other threat that could be affecting the network. Every device that is added to the network, every change in the topology or any failure needs to be immediately detected. That is why we need a network monitoring software to make the best of our network and that is exactly what Nectus is for.

Nectus is a network discovery, monitoring and visualization software and its main role is to discover network topology, generate a visual network diagram and keep it up to date, detecting any failure or unusual behavior that could be affecting the network and alerting the network administrator immediately. 

These are the Nectus key features: 

· Automatically discovers connections between devices (via Cisco Discovery Protocol) and stores all this connections in a Database.

· Network discovery is run everyday and if any new device has been added to the network, the topology is updated automatically so network diagrams will always be up to date, no need to do it yourself so you will be saving your time. You can generate L2 and L3 network topology in just one click, way better than Microsoft Visio!

· Real time monitoring is overlaid on top of network diagrams. You will be able to see utilization, errors and dropped packets or up/down status directly on your diagrams. 

Read more...Collapse )

Network Visualization Made Simple

What is the first question you ask when you start a new network project or start a new job as a network engineer?":

- Can I see your network diagram?

Lucky you if you get it right away and in the "right" format (Visio?) but in my past I remember places where it took me weeks to find the right person who had that diagram,

which was very often not up to date or did not contain information I needed or was only showing application layer components and I still had to spend hours doing "show cdp nei"

and re-creating drawings the way I like it with the information I needed.

I always wished there was a magic tool where I can just click one button "Generate Topology" and get it auto-generated with the information I wanted to see.

Shouldn't be that hard to do, right? Well, finally I found a perfect one.

I got a demo copy of Nectus from www.nectus5.com and its simplicity and effectiveness made it my number one network software tool (ahead of my long standing love for SecureCRT).

The main reason for its beauty is integration with Network Discovery functionality.

Nectus does discover all the devices and all the interconnections, and uses that information when generating a network topology. As a Cisco-centric tool, it relies on CDP to build its interconnection dataset so one of the main requirement is operational SNMP and CDP protocol on every link you want to see on your network diagram. But that is the only manual work you have to do. After initial configuration is done, the only part that separates you from your network diagram is a single click of a button. Right click on any device in GUI and select "Create a Network Topology" - BOOM!

It took 10 seconds to generate complete topology of our Datacenter of 350 Nexus switches with all the interface names and IP addresses, and I could even overlay real-time traffic utilization info over each link.

No more aligning shapes in Visio (even though you can export Nectus diagrams to Visio). No more changing font size one line at a time. No more hard manual work.

Here is an example of what one button click can produce:

I didn't have to worry about keeping track of changes: the Discovery module is configured to run every day and all the changes are automatically stored in DB so I just have to "Refresh" existing topology.... ah forgot to say.. and all of these features are in your browser. No software clients to install. I use it every day now. Honestly.

(no subject)

горох в техасе  .,.  сразу вспоминаю, икра баклажанная .. заморская



Palo Alto vs. Cisco (6) NAT Description
возможность добавить Description к NAT rule в PaloAlto это счастье.
это сможет понять только тот, кто хоть раз попытался сделать 200 IPsec tunnels на одном ASA

Cisco vs. Palo Alto (5) DHCP Reservation
ладно, будем объективными..  PaloAlto умеет делать DHCP Reservations..   :)
Cisco почему-то упорно игнорирует запросы маленьких контор где нет dedicated DHCP server..

PaloAlto vs. Cisco (4) Dashboard
Смотрю я на этот PaloAlto Dashbord и думаю, что такого полезного можно увидеть здесь?


Да ничего! Я им что, должен все интерфейсы наизусть помнить?
Мне надо видеть логические имена типа: Inside, Outside, DMZ...
вот уроды блин...

PaloAlto vs. Cisco (3) XML sucks
ну какой идиот в Пало Алто придумал хранить configuration v XML?
надо делать продукт не как удобно программистам, а как удобно юзерам.

Palo Alto vs. Cisco (2)
2. GUI не преспособлен для работы с multi-level nested groups с количеством уровней >= 3.

предположим такую не децкую ситуацию когда у нас есть 3 вложенные группы
Group1 -> Group2 -> Group3 .
Юзер реально помнит только самую верхнюю группу Group1  и захотел прогулятся по дереву ..
идем в Objects-> Address Groups и


нажимаем на Group1, она открывается и мы видим Group2.

Все отлично.  Дальше нажимаем на Group2 она открывается и мы видим ...


все что угодно но только
не Group3.  Что с этим делать реално не понятно..  ето такое фундаментальное ограничение
У меня на Циске около 5,000 групп и вложения бывают до 4-го уровня..