Packet Deduplication Generator TAP for Network Visibility and Data Optimization
Data De-duplication is a popular and popular storage technology
that optimizes storage capacity.It eliminates redundant data by
removing duplicate data from the dataset, leaving only one copy.As
shown in the figure below.This technology can greatly reduce the
need for physical storage space to meet the growing demand for data
storage.Dedupe technology can bring many practical benefits, mainly
including the following aspects:
| (1) | Meet ROI(Return On Investment)/TCO(Total Cost of Ownership)
requirements; |
| (2) | The rapid growth of data can be effectively controlled; |
| (3) | Increase effective storage space and improve storage efficiency; |
| (4) | Save the total storage cost and management cost; |
| (5) | Save the network bandwidth of data transmission; |
| (6) | Save operation and maintenance costs such as space, power supply
and cooling. |
Dedupe technology is widely used in data backup and archiving
systems, because there are a lot of duplicate data after multiple
backups of data, which is very suitable for this technology.In
fact, dedupe technology can be used in many situations, including
online data, near-line data, and offline data storage systems. It
can be implemented in file systems, volume managers, NAS, and
sans.Dedupe can also be used for data disaster recovery, data
transmission and synchronization, as a data compression technology
can be used for data packaging.Dedupe technology can help many
applications reduce data storage, save network bandwidth, improve
storage efficiency, reduce the backup window, and save costs.
Dedupe has two main dimensions: deduplocation ratios and
performance.Dedupe performance depends on the specific
implementation technology, while Dedupe rate is determined by the
characteristics of the data itself and application patterns, as
shown in the table below.Storage vendors currently report
deduplication rates ranging from 20:1 to 500:1.
| High deduplication rate | Low deduplication rate |
| Data created by the user | Data from the natural world |
| Data low rate of change | Data high rate of change |
| Reference data, inactive data | Active data |
| Low data change rate application | High data change rate application |
| Full data backup | Incremental data backup |
| Data long-term storage | Data short-term storage |
| Wide range of data applications | Small range of data applications |
| Continuous data business processing | General data business processing |
| Small data segmentation | Big data segmentation |
| Elongate data segmentation | Fixed length data segmentation |
| Data content perceived | Data content unknown |
| Time data deduplication | Spatial data deduplication |
Dedupe Implementation Points
Various factors should be considered when developing or applying
Dedupe technology, as these factors directly affect its performance
and effectiveness.
| (1) | What | What data are de-weighted? |
| (2) | When | When will the weight be eliminated? |
| (3) | Where | Where is the weight elimination? |
| (4) | How | How to reduce the weight? |
Dedupe Key Technology
Deduplication process of storage system in general is this: first
of all the data file is divided into a set of data, for each block
of data to calculate the fingerprint, and then based on fingerprint
Hash search keywords, matching indicates the data for the duplicate
data blocks, only stores data block index number, otherwise it
means the data block is the only piece of a new, storage of data
block and create relevant meta information.Thus, a physical file in
the storage system corresponds to a logical representation of a set
of FP metadata.When reading the file, first read the logical file,
then according to the FP sequence, take out the corresponding data
block from the storage system, restore the copy of the physical
file.It can be seen from the above process that the key
technologies of Dedupe mainly include file data block segmentation,
data block fingerprint calculation and data block retrieval.
(1) File data block segmentation
(2) Data block fingerprint calculation
(3) Data block retrieval
What's the Network Packet Broker and Network Taps to do?
With the acceleration of carrier broadband construction and the
rapid development of 3G, 4G, and 5G networks, and with the
popularity and development of big data and cloud computing
technologies, data traffic is growing, and the bandwidth
requirements for backbone networks and core networks are
increasing. Since 2013, the major operators have begun to gradually
upgrade the backbone network. The core backbone network
transmission link media starts to be expanded from 10GPOS and
40GPOS links to 100G Ethernet links. It will inevitably lead to the
continuous updating of big data collection and analysis technology.
In the face of 100GE link environment with high bandwidth capacity,
how to effectively meet the requirements of various national
security, network security monitoring, operator intelligent
pipeline DPI data collection and analysis will become a hot spot in
the field of data acquisition and analysis.
Chengdu Shuwei Communication is following the development direction
of the Internet and develop the MATRIX-TCA-CG Traffic
Acquisition(Traffic Capture) Visibility System, and dedicated to
the traffic collection and visual analysis application requirements
on 1GE, 10GE, 10G/40G POS, 40GE and 100GE links. The MATRIX-TCA-CG
integrates free-steering output functions such as network flow
collection/acquisition, aggregation, filtering,
forwarding/distribution, and load balancing and provides an
efficient solution for the flow analysis.
Matrix-TCA-CG is a network data visualization control device for
high density and large flow of 10G/40G/100G.
Matrix-TCA-CG is specially designed for the data collection of 10GE
40GE and 100GE links distributed intensively in operators mobile
Internet export, IDC export, provincial network and backbone.
Based on ATCA standard architecture, MATRIX-TCA-CG has high
carrier-level reliability and strong expansibility which can
smoothly configure the functional board CARDS to cope with port
demand scenarios of various specifications and sizes.
SDN Network Packet Broker Applications:
After sorting out the main participating entities of the network,
the application scenarios of SDN basically focus on telecom
operators, government and enterprise customers, data center service
providers and Internet companies.The application scenarios of SDN
mainly focus on: data center network, interconnection between data
centers, government-enterprise network, telecom operator network,
and business deployment of Internet companies.
| Scenario 1: | application of SDN in data center network |
| Scenario 2: | application of SDN in data center interconnection |
| Scenario 3: | application of SDN in government-enterprise network |
| Scenario 4: | application of SDN in telecom operator network |
| Scenario 5: | application of SDN in the service deployment of Internet companies |
Network Packet Broker Specification and Module Type
| NT-FTAP-32QCX NetTAP® TAP/NPB Functional Parameters |
| Network Interface | 100G(compatible with 40G) | 32*QSFP28 slots |
| Out band interface | 1*10/100/1000M cooper |
| Deploy mode | Fiber tap | Support |
| Mirror span | Support |
| System function | Traffic processing | Traffic replicating/aggregating/splitting | Support |
| Load-balancing | Support |
| Filter based on IP/protocol/port quintuple traffic identification | Support |
| UDF Matching | Support |
| Single fiber transmission | Support |
| Ethernet package independence | Support |
| Processing ability | 3.2Tbps |
| Management | CONSOLE MGT | Support |
| IP/WEB MGT | Support |
| SNMP MGT | Support |
| TELNET/SSH MGT | Support |
| SYSLOG Protocol | Support |
| RADIUS or AAA Centralized authorization | Support |
| User authentication | Authentication based on username and password |
Electrical (1+1 Redundant Power System-RPS) | Rated power supply voltage | AC110~240V/DC-48V[Optional] |
| Rated power frequency | AC-50HZ |
| Rated input current | AC-3A / DC-10A |
| Rated function power | Max 450W |
| Environment | Operating temperature | 0－50℃ |
| Storage temperature | -20-70℃ |
| Working humidity | 10%-95%, No condensation |
| User Configuration | Console Configuration | RS232 interface,115200,8,N,1 |
| Password authentication | Support |
| Chassis Height | Rack Space (U) | 1U 445mm*44mm*505mm |
NT-FTAP-32QCX
Network TAP NPB.pdf
