IPSec is basically a way to secure the data transfer between computers. IPSec is making data traffic between two nodes secure by making these things:
- Data Authentication
- Authentication – The packets can be spoofed and not sent from the place they seem they are sent. Data origin authentication can be provided with configuration of IPSec. IPSec will enable us to ensure that received packet from a trusted party is really originating from that party.
- Data integrity – IPSec can be configured to secure that the data packet are not changed when they are crossing the network.
- Anti-replay protection – IPSec is able to check that the packets received are not duplicates of previous data packets.
- Encryption
- IPSec is a technology that enables you to encrypt network data so that it cannot be captured and used by unauthorized persons.
For example, in Windows Server 2008, IPSec is enabled by IPSec Policies or sometimes Connection Security Rules. IPSec Policies are in charge of negotiation for authentication and encryption. In other side, Connection security rules will only try to negotiate authentication services by default.
IPSec Policies
IPSec Policies define how computers are dealing with IPSec communications. In Microsoft world you can define an IPSec Policy to an individual computer by using Local Security Policy or you can set the IPSec policy to a group of computers by Group Policy.
IPSec Policy is somehow similar to Access Lists in networking world. It is composed of one or more IPSec Policy rules that are saying how IP traffic should be protected and by which means. Each policy rule is triggered by IP filter list. IP filter list can be seen as Route-Mapping system in networking for the way of “caching” events in the traffic circulation.
IP filter lists contain a set of IP filters that can match IP traffic and apply the IPSec Policy to that traffic. IP filters define a source or destination address, range of address, computer name, TCP/UDP port, or server type like DNS, WINS, DHCP, etc. If traffic entering or leaving the computer on which a policy is assigned matches a filter, the filter action associated with that rule is applied. Filter actions for some rule are block, permit, or negotiate security.
Example for IPSec Policy 
We can see from the picture that IPSec Policy is made of rules, filters, and filter actions. IPSec Policy is made up of three rules.
- Most specific rule and that’s way the first one it defines traffic by type and by address. We can see that the type is defined as Telnet or POP3 and that the addresses are 192.168.3.32 and 192.168.3.200.
- The second rule is the next most specific one. This rule defines traffic by type, Telnet or POP3.
- The third rule is the less important because it applies to all traffic.
If we think about the result, IPSec policy will attempt to authenticate, not encrypt, all data exept data for Telnet and POP3. Data for Telnet and POP3 are blocked if they are coming from 192.168.3.32 for Telnet and 192.168.3.200 for POP3. In this case the data will be forwarded only if encryption if successfully negotiated.