ChangeLog

ChangeLog
v0.0.1 - PAT - Pktgen Automation Tool
	 Getting port stats is now possible

v0.0.2 - different packet sizes can be set for measurements

v0.0.3 - measurement results are written into output files

v0.0.4 - configuration needs to be done in a separate config.cfg file

v0.0.5 - if output file exists, no header will be written again -> this helps
easier parsing later (for instance gnuplot, or shell scripts)

v0.1.0 - Separated BASH application with more than one module for analyzing

v0.1.1 - Add feature of analyzing the res files - calculating avg, min and max
values

v0.1.2 - main.sh creates now symlinks for pat.cfg and PAT.lua in pktgen's root
dir. User do not have to make it on his or hers own.

v0.1.3 - Refined gnuplot output file name. Add feature of setting the bps unit as
well. Now, accordingly PAT calculates correctly the throughput in the given unit
and presenting via gnuplot also uses this information.

v0.1.4 - Remove Miss, Diff from presentation and creating a separate charts for
them. Setting PKTGEN_BINARY is now possible in the main.cfg

v0.1.5 - Feature of setting min and max outlier can be defined in main.cfg

v0.1.6 - Simplify code and add feature of calculating and plotting the theoretical results as well.

v0.1.7 - Change name from PAT to A2NFPA (Advanced and Automated Network Function
Performance Analyzer)

v0.1.8 - You can set your measurement scenario name via the first argument.
This scenario name will help you identify results files later.

v0.2.0 - Make A2NFPA Python-based instead of BASH
	 Only one config file exists! No possibility for causing inconsistency
	 Pktgen config file are created during running the app according to the desired preferences
	 Traffic types are now recognized and if a pcap file exists for a given traffic type then
	 it is going to be used and measured

v0.2.1 - Old A2NFPA is fully migrated to python-based code - OLD FUNCTIONS work
as they worked formerly. Do not use new ones yet. APP_NAME from config is purged.
App name is going to be the same in the future!

v0.2.2 - using PCAP files are working! It is easy to create, define and use your
own pcap file, just follow the naming conventions. Let's say you have a special
pcap file where only 67 bytes packets are present, and you call it mypcap.
After this, you only need to rename your pcap to a2nfpa.mypcap.67bytes.pcap and
place it in PCAP folder. Then in a2nfpa.cfg set packetSize=67, and
trafficType=mypcap.
Only specific packet sizes are working!!!! More realistic pcaps are not supported!

v0.2.3 - BiDirectional support for simple traffic!
Now setting up biDir=1 in a2nfpa.cfg enables to generate traffic in both directions
and also capturing port rate is working! Direction is indicated now in the gnuplot/
res filenames as well (uniDir|biDir)
to send traffic

v0.2.4 - BiDirectional support for special traffic!

v0.2.5 - Realistic PCAP file support is now available. It is possible to
set a special traffic file, for instance, WIFI for measurement.
A2NFPA will search for a pcap file in MAIN_ROOT/PCAP folder named a2nfpa.WIFI.pcap!
So, name your pcap properly and set in a2nfpa.cfg (realisticTraffic=WIFI)
New gnuplot file templates are used for realistic traffics

v0.2.6 - Bidirectional Realistic PCAP file support is available. Usage didn't
change.

v0.2.7 - The preset PKTGEN core_mask and cpu_core_assign variables are now
checked before start, thus A2NFPA will realize if those settings are invalid.

v0.2.8 - Capable of using different SYNTHETIC pcap files for different ports at
the same time. Read a2nfpa.cfg to know how to use it! Remark:
"#If two traffic types are separated via 1 'PIPE (|)', then two different pcaps
#will be loaded to the two ports, so bidirectional measurement will be done
#for this traffic type irrespectively whether the latter biDir property is 1 or 0.
#On the other hand, biDir property works the same as it worked for other traffic
#types"

v0.2.9 - Capable of using different REALISTIC pcap files for different ports at
the same time. Works the same as SYNTHETIC works

v0.2.10 - Results files (.data) are comma separated as well in order to easily
import it to, for instance, Libreoffic Calc or MS Excel for easier reading.

v0.2.11 - Bug fix for plotters (handling commas in gnuplot charts properly, and
calculate/show throughput units (pps,bps) properly

v0.2.12 - Patch for plotter bugs


v0.3.1 - A2NFPA saves results to (remote PostgreSQL) database. This feature is
temporary since it's unsecure.
The old visualizer and .data files are also supported and generated.
The database has now more columns, so a2nfpa.cfg has also changed. New HW/SW
related variables needs to be filled. And user information is also necessary

v0.3.2 - Logs are now saved into MAIN_ROOT/log. Estimate time calculator is
added to show at the beginning how much to do we have for a coffee break.
Database is also updated for showing whether a measurement was solely uniDir or
biDir. bidir_twin_id is insufficient to indicate this.

v0.3.3 - Improved Estimated time calculation

v0.4.1 - Changed name to NFPA. Minimal web-gui features added.
Added more arguments and change argparser. Now, it should be used:
sudo python nfpa.py [-h] -n NAME [-w WEB]


v0.4.2 - Reading configuration file is now separated in order to be used from
other modules.

v0.4.3 - Create proper Web-GUI fillout form for configuration.

v0.4.4 - Add 'floating' <div> elements to configuration form to help the user.

v0.4.5 - Add configuration comments to source in order to use it consistently.

v0.4.6 - Refine CLI arguments to define web-gui which ip/host and port should
be used.

v0.4.7 - Remove port_mask and promiscuous_mode from config, since they are always
need to be the same (port_mask = 3, and -P for promiscuous in order to capture
any packet)

v0.4.8 - Make SQLite Database path static, instead of configurable.

v0.4.9 - Configuring via CLI or WEB-GUI produces the same configurations, and
config file is always consistent and represents the last setup.

v0.4.10 - Refactor everything from a2nfpa* to nfpa*

v0.4.11 - Add confirm dialog to web-gui to ask again about enabled DPDK interfaces

v0.4.12 - Minor bugfix for handling packetSize,trafficType, and realisticTraffic
properties to be empty.

v0.4.13 - Bugfix for the cli version initiated by the new argument parser and
structure

v0.4.14 - Correct estimate time left calculation for realistic traffics

v0.4.15 - Measurement progress bar is added to WEB_GUI

v0.4.16 - Refine GUI and use special logger class to WEBNFPA class as well.

v0.4.17 - Temporary get back to use pcap files starting with a2nfpa instead of
nfpa.

v0.4.18 - Minor changes in help. After starting web gui on a certain ip:port,
then NFPA can be accessed via http://ip:port/nfpa

v0.4.19 - Minor changes in help and GUI to indicate the usage in a more easier
way.

v0.4.20 - Bugfix for remote PostgreSQL connection. Now, there is a timeout for
connection, and if connection is not working, then NFPA will exit. If connection
is OK, then NFPA disconnects from the database, and connect later when needed.
This is useful, since a measurement can last to many hours, and database
connection will timeout if it has been created at the beginning.

v0.5.1 - Migrate database handling to local postgres

v0.5.2 - Merge last bugfix of v0.4

v0.5.3 - Add port_mask setting possibility, since it is possible to use only 1
port for both directions. Accordingly, add checking for this parameter.

v0.5.4 - Add some error propagation to WEB-gui in order to not just indicate
errors in the terminal, but in some cases the WEB_gui also shows some basic
information

v0.5.5 - If configuration errors happen, WEB-gui does not exits. It is possible
to go back and reset some misconfigured parameters. Note that in case of system
related errors (e.g., directory not exists), it is not supported (yet)!

v0.5.6 - Add some views (defined in postgresql) to sqlite database and do some
minor changes in sqlite database to make it consistent with postgresql

v0.5.7 - Add testing feature that could be set via -t option by starting NFPA.
This indicates that no PKTGEN will be started and a pre-measured test data (.res
files) will be read and processed. Useful feature for testing purposes. The
nfpa.cfg file MUST contain in this case no realistic trace, and only simple
traffic trace with packet sizes 64,128,256,512,1024,1280 and 1500, respectively.

v0.5.8 - Change Postgres SQL adapter to sqlite. Now, results are saved in local
sqlite database.

v0.5.9 - Add an original empty sqlite database file to the repository

v0.5.10 - Refine Gnuplot scripts to create IEEEtran, two-sided paper compatible
output where the font sizes match the text size

v0.5.11 - Add proper bidir column handling in sqlite database handler.

v0.5.12 - Repair sqlite database schema: wrong foreign keys were used in
measurements table

v0.5.13 - Refine res directory handling. Now, a not so deep  granularity is
also provided during creating res dirs (vnf_name/driver/cpu/virt/port_type/dir/)

v0.5.14 - Use own invoker in many opportunities. This helps to log errors to
log files

v0.5.15 - Minor bugfixes and set pktgen running back to os.system() in order to
be able to watch the processing in the terminal

v0.5.16 - Measurement webpage now shows the path to logfile that is being
generated

v0.5.17 - Minor changes to GUI

v0.5.18 - Add some info about new rules of using vnf_function parameter as it
should be set corresponding to the VNF, i.e., if formerly the port forward
string was set as comment, then one must use bridge as vnf_function. Accordingly,
if l2-switch is used, then always install proper flow rules in your vnf, i.e.,
it will mean the DMAC learning formerly set as comment parameter.
Furthermore, DON'T USE trxE and trXI traces at the same time to avoid ambiguous
results.

v0.5.19 - Change pcap file names from a2nfpa.* to nfpa.*

v0.5.20 - Refine some infos in config file and web gui

v0.5.21 - Add temporary pktgen .res files deletion at the beginning of the
processing as well. This will clean up pktgen's root, and those files that are
existing there due to a crashed/cancelled process of NFPA cannot influence
the measurements results

v0.5.22 - Remove 'test' option completely.

v0.5.23 - Config parameter values can now contain '=' signs as well. It could
be useful, for instance, if OpenFlow flow rules wanted to be set as vnf_comment
parameter

v0.5.24 - Set gnuplot property terminal from enhanced to noenhanced in order to
make text as underscored instead of subscript.

v0.5.25 - Correct one port support! NFPA did not recognized correctly the
configuration if only 1 port wanted to be used instead of the 2-port setup.

v0.5.26 - Add storePID function to store NFPA's PID in nfpa.pid file located
under NFPA's MAIN_ROOT. This helps to easily kill the whole process tree during
execution if needed.

v0.5.27 - Add NFPA killer script (kill_nfpa.sh), which kills the whole process
tree of NFPA.

v0.5.28 - Minor bugfix

v0.6.1 - NFPA is compatible with the newest version of PktGen 2.9.12, and
since there is no more long lasting ads and information when starting pktgen
the running time has been reduced, and the remaining time calculation has
been adjusted accordingly. Thus, NFPA recommends to use PktGen 2.9.12, which
perfectly works with DPDK 2.2.0.

v0.6.2 - NFPA is now capable of creating the SQLite database file from scratch!
Accordingly, you do not need to download the nfpa.db file separately, if you want
to have a clear database. In this case, just remove/delete nfpa.db from the db/
directory, and the first run of NFPA after removal will create a new and
empty database!

v0.6.3 - Minor bugfixes around scenario name parsing and creating database from scratch

v0.7 - Full fledged multi-core support (now core-mask check function allows to define mulitple
cores separately for RX and TX queues)!

v0.8 - Add INFINITE LOOP capability. By setting measurementDuration in nfpa.cfg to 0,
the first proper traffic type and trace will be run forever, until NFPA
becomes killed (to this, use kill_nfpa.sh helper script).

v0.8.2 - Lua scripts now writes out the temporary results into the temporary
result files in each seconds instead of writing them at the end of the
measurement. This is useful if NFPA is running in an infinite loop, and
a third-party application may want to observe the actual throughput data.

v0.8.3 - Minor bugfix to lua scripts

v0.9 - NFPA now supports DPDK parameter --socket-mem. This way, it is possible to avoid consuming
all the hugepages we have. Moreover, NFPA checks in advance whether there are sufficient hugepages
before the much slower pktgen initialization would print out the corresponding information

v1.0 RC1 - New version of NFPA is available. It is still in RC1 phase, but the reason, why it became
v1.0 is that now we are working on to make NFPA to be run in virtualized environments, and provide a
NFPA-as-a-service function. The new thing is basically the 'other_dpdk_params' in the nfpa.cfg,
where you can set any further parameters that may be required to run NFPA efficiently in your environment.
In particular, it could be settings for configuring vhost interfaces for pktgen.

v1.0 RC1 - socket_mem argument became unnecessary to be set to ease the setup if one not knows
the amount of hugepages his or her setup needs, just want to use all without calculating this
number.

v1.0 RC1 - Add -r,--reset runtime argument. This argument disables terminal reset, which is now enabled by default!
This means that NFPA repairs the messed up terminal after measurement is done, and there is no need for logout and
login again.

v1.0 - Test confirmed that NFPA works well with the latest features, and can run in docker containers, or othe VMs using
vhost interfaces. In docker, we measured that the packet sending rate using OVS 2.5.0 with DPDK could as high as 6 Mpps
(64-byte packets), but in most cases when NFPA was running in the container, the receiving rate was about 4 Mpps.
On the other hand, with the new 'other_dpdk_params' configuration parameter makes it possible to run several NFPA
applications at the same time on the same machine! To do this use DPDK's --file-prefix parameter. For more info, check
http://dpdk.org/doc/guides/prog_guide/multi_proc_support.html#running-multiple-independent-dpdk-applications.

----------=================== NFPA v2 SpongeBob ==================------------
About versioning and naming convention...well, NFPA is evolving so fast, we need to name each greater
major revisions. We follow similar naming conventions others have used (like Android) but it will be very close
to Debian Linux's naming conventions, where the different versions have the name of the (almost randomly chosen)
characters from the movie Toy Story.
We stick to Spongebob Squarepants instead! Since current version makes (or will do) much more stuffs for you as the
previous version, we named it after protagonist Spongebob, who is a workaholic.
So, behold and please welcome NFPA V2 Spongebob! Logo update is coming soon...


v2.0 alpha
Add configuration support for remote VNF configuration - currently only openflow is supported.
Flow rules are assembled to use them for remote configuration. In case of OpenFlow switch,
make sure to start it with a passive TCP port for the remote configuration.
Remote configuration also can be done via WebGUI.


v2.1 alpha
Now the the in- and output ports could be setup in the nfpa.cfg or via the WebGUI. It is useful for different
controller application and APIs since these setups will replace meta-ports in the pre-defined flow rule files.

v2.2
Flow rules preparator is added, which parse the basic flow rules files containing meta port information
properly to the one the required configuration needs. NFPA saves theses files under of_rules/tmp with
timestamps in their names, so they could be found easily and the whole thing won't mess up your directory
tree.

v2.2 beta
Initial tests have confirmed that the NFPA can easily configure remote openflow switches with
the predefined scenarios.
Estimated Time Left calculation is also updated by the fact that a dpdk application always zeros
all the avaialble hugepages even if you only want to use just a couple of them. Now, the number of
hugepages are also taken into account when estimated time is calculated.

v2.2.1 beta
Updated l2-switch scenarios have been added to of_rules directory. Now, they are more realistic.
More information can be found on the website.

v2.5 beta
Greater minor version jump, since fundamental parts have been changed. NFPA now also stores in the 
database (via the nfpa.cfg) the number of cpu cores the VNF has used, therefore it becomes much
simpler to filter the measurements data accordingly.

v2.6
SQLite and the main database is also altered to store control_nfpa variable as well in order to
indicate whether the supported scenario with a given trace is configured via NFPA.

v2.7
NFPA went through a great code refactoring to solve one of the most important issue as the measurement
data was only processed and analyzed after all measurement were done. However, if some error occurred
before the last measurement, for instance, someone has consumed the hugepages from Pktgen, then the
results measured so far have not been analyzed, plotted and inserted into the database.
Now, all the measurement results are immediately analyzed trace by trace, i.e., if one measurement was
completely successful, then its results will not be vanished.

v2.7.1
vnf_num_cores parameter could not be set via webGUI. Now, it supports it.

----------=================== NFPA v3 Patrick ==================------------
v3.0 - The new version of NFPA provides more lazyness for the user, that's why the new version's
name comes from Patrick Star, the starfish who lives under a rock, and his intelligence is extremely
low. However, as a really good (but not necessary) extension, we named the new version after him as
he is the best friend of Spongebob.
About the feature: An email sending service has been developed, which when enabled could connect to your SMTP server with
STARTTLS security (like Gmail), and once NFPA finished with one measurement it
immediately sends you an email about its status, moreover, it attaches the log file,
and all the related result files.

v3.0.1 - NFPA logo is added to webGUI, and some usability issues has been worked around in webGUI, 
for instance, when a feature is disabled the corresponding elements became disabled as well, i.e.,
they became uneditable.

v3.0.2 - Bugfixes

v3.0.3 - Bugfixes

v3.0.4 - Modify email sending service - now the SUBJECT also reflects some details, and ERROR message also contains
the measurement setup data.

v3.1 - New starting argument has added. If someone does not want to store the measurements data in the database, since
they are just for testing purposes and the user does not want to mess up the database with test results just use -d, or
--nodatabase starting parameter to reach this end.

v3.2 - New starting parameter has added for defining the path to the config file (nfpa.cfg). Now, it became easier
to automatically start NFPA multiple times consecutively for different use-cases, e.g., for different VNFs, by defining
multiple config files. See the output of python nfpa.py -h

-- CBT FORK
v3.3 - Added access gateway use case that aggregates subscribers located behind customer endpoints to the Internet.
It consists of a Virtual Provider Endpoint (VPE) that serves Internet access to subscribers located behind Customer
Endpoints (CEs). For brevity, we identify CEs with the MAC address and we assume that the operator sets 10 CEs, each
serving 20 users provisioned with a private IP address that is unique within the CE. Table 0 separates user-to-network
traffic on a per-CE basis from network-to-user traffic; user-to-network traffic goes to per-CE flow tables
(table 11, etc.) that match the source IP address to identify the user and rewrite the source address to a unique public
IP address (realizing a NAT), and finally to the IP routing table (table 200) that contains 10K random IP prefixes;
packets from yet unseen users will in turn be forwarded to the controller to perform admission control and allocate a
unique IP address. Network-to-user traffic on the other hand goes to table 110, which matches on the destination IP to
identify the user and the CE, swaps the destination IPs back to the private IP addresses, and then sends packets
to the proper CE.
Use vnf_function=AGW_10CE_20UserPerCE_10kfib and use trafficType=AGW_[X]flows, where [X] = {1,10, 100, 1000, 10.000,
100.000}. - the compressed PCAP files could found under new_PCAP/ directory (nfpa.agw_unidir.tar.xz)

v3.4 - Add hungarian language support for gnuplot plots. It can be set in nfpa.cfg under gnuplot related settings.

v3.4.1 - Minor changes to nfpa_traffic.lua. Originally one measurement lasted until the trace with all of its packetsizes
has been measured. However, since results are being analyzed for each traffic trace (consisting only one size of packets
at a time), some codes became unnecessary.

v3.5 - Multiple language support at the same time. Now, several plot_language lines could be defined (currently 2 only
due to the number of supported languages so far), and for the measurements the plots will be generated in all
languages

v3.6 - New initializing argument. Now, it is possible to disable plot generation, and only produce the result files (.data).
This could be useful, when you don't have the right version of gnuplot, or just don't need the plots as other scripts
you have written is parsing the result files later anyway. Doesn't matter for what reason it is good or not, but now it
is switchable :)

v3.7 - Support measuring BESS datapaths (Berkeley Extensible Software Switch,
  https://github.com/NetSys/bess/).  See documentation in lib/plugin/bess.py

v3.8 - New way of configuration!
NFPA's config file handling has been changed:
Now, the originally big one config file has been divided into smaller pieces (e.g., [YOUR_CONFIG].email,
[YOUR_CONFIG].nf_ctrl, [YOUR_CONFIG].traffic) in order to make a higher granularity level for the configuration.
Each simpler config file is representing an important part of the whole configuration. This makes it easier to change
just one relevant part, for instance, the traffic generation part without opening a big file and looking for the right
line. Furthermore, sharing configuration has also become much easier; only the relevant part needs to be shared without
forcing the other party to change manually the other parts to his/her own environment (e.g., username, nf_hw).
See nfpa.cfg.README before using the new version.