hale
Published on 2024-12-24 / 33 Visits
0
0

SRv6 BE TE配置及测试

一、实验目的

通过LAB了解SRv6基本概念、SRv6 BE、SRv6 TE Policy、SRv6 TE Flow Group工作机制、基本配置。

二、实验环境

2.1 实验拓扑

2.2 数据准备

2.2.1 IPv4/IPv6地址

IPv4

IPv6

Loopback0地址

PE1

192.168.1.1/32

2001:1::1/128

P2

192.168.2.1/32

2001:2::1/128

P3

192.168.3.1/32

2001:3::1/128

PE4

192.168.4.1/32

2001:4::1/128

设备互联地址

设备之间互联地址

192.168.AB.0/24
比如PE1和P2互联地址分别为:192.168.12.1和192.168.12.2

2001:AB::/64
比如PE1和P2互联地址分别为2001:12::1和2001:12::2

终端地址

CE1

172.10.1.10/24

172.10.1.10/24

CE2

172.20.1.10/24

172:20:1::10

2.2.2 SRv6 SID 地址

PE1

P2

P3

PE4

Locator

3001:1::/80

3001:1::/80

3001:1::/80

3001:1::/80

Locator name

PE1

P2

P3

PE4

End

::10

::10

::10

::10

End.X

::AB(注)

::AB

::AB

::AB

注:比如PE1上,针对PE1和P2互联端口End.X为::12

End.DT4

::4001

:4001

End.DT6

::6001

::6001

End.OP

::7001

::7001

::7001

::7001

2.2.3 VPN规划

RD

RT

vpn1

PE1:100:1

PE4:100:2

100:100

三、 实验内容

  • 基础配置:包括端口、ISIS、VPN、BGP、SRv6基础配置

  • SRv6 BE测试

  • SRv6 TE Policy测试

  • SRv6 TE Flow Group测试

四、基础配置

4.1 端口基础配置

配置各接口的IPv4/IPv6地址。端口配置IPv4地址非必须,可以不配。

PE1配置,其他类似:
#
Sysnam PE1
#
int lo0
ipv6 enable
ip add 192.168.1.1 32
ipv6 add 2001:1::1 128
#
int ether3/0/0
ipv6 enable
ip add 192.168.12.1 24
ipv6 add 2001:12::1 64
#
int ether3/0/1
ipv6 enable
ip add 192.168.13.1 24
ipv6 add 2001:13::1 64
#
 
检查命令:
display ip interface brief
display ipv6 interface brief

4.2 ISIS基础配置

ISIS路由器的network-entity(xx.xxxx.xxxx.xxxx.xxxx.00)可自行定义,这里采用loopback0地址填充为system id,比如loopback地址是192.168.1.1,则系统ID:1921:6800:1001。

PE1配置,其他类似:
#
isis 100
 is-level level-2
 cost-style wide
 network-entity 49.0001.1921.6800.1001.00
 is-name PE1
 #
 ipv6 enable topology ipv6
#
int lo0
isis enable 100
isis ipv6 enable 100
#
int ether3/0/0
isis enable 100
isis ipv6 enable 100
isis circuit-type p2p
#
int ether3/0/1
isis enable 100
isis ipv6 enable 100
isis circuit-type p2p
#
 
检查命令:
display isis peer
display isis route
display ip routing-table

4.3 VPN基础配置

ENSPPro模拟软件比较消耗内存资源,为了节约电脑内存资源,简化CE侧的接入,终端直接接入PE。

在PE设备上配置VPN实例并绑定到接入侧接口。同一个VPN实例下配置IPv4和IPv6 地址簇,允许VPN内IPv4和IPv6地址接入。

PE1配置:
#
ip vpn-instance vpn1
 ipv4-family
  route-distinguisher 100:1
vpn-target 100:100 both evpn
 ipv6-family
  route-distinguisher 100:1
vpn-target 100:100 both evpn
#
int ether 3/0/2
ip binding vpn-instance vpn1
ip add 172.10.1.1 24
ipv6 enable
ipv6 add 172:10:1::1 64
#
PE4配置:
#
ip vpn-instance vpn1
 ipv4-family
  route-distinguisher 100:2
vpn-target 100:100 both evpn
 ipv6-family
  route-distinguisher 100:2
vpn-target 100:100 both evpn
#
int ether 3/0/0
ip binding vpn-instance vpn1
ip add 172.20.1.1 24
ipv6 enable
ipv6 add 172:20:1::1 64
#
 
 
检查命令:
display ip vpn-instance verbose

4.4 BGP基础配置

P2、P3配置为RR,设置RR Cluster ID 100。

采用Peer Group的配置形式。

PE1配置,PE4类似:
#
bgp 100
 router-id 192.168.1.1
 undo default ipv4-unicast
 group RR internal
 peer RR connect-interface LoopBack0
 peer 2001:2::1 as-number 100
 peer 2001:2::1 group RR
 peer 2001:3::1 as-number 100
 peer 2001:3::1 group RR
 #
 l2vpn-family evpn
  policy vpn-target
  peer RR enable
  peer 2001:2::1 enable
  peer 2001:2::1 group RR
  peer 2001:3::1 enable
  peer 2001:3::1 group RR
 #
 ipv4-family vpn-instance vpn1
 #将vpn端口直连路由引入为bgp vpnv4路由
  import-route direct
 #将vpnv4路由转为evpn type 5路由发布给evpn邻居
  advertise l2vpn evpn
 #
 ipv6-family vpn-instance vpn1
  import-route direct
  advertise l2vpn evpn
 
P2(RR)配置,P3(RR)类似:
#
bgp 100
 router-id 192.168.2.1
 undo default ipv4-unicast
 group PE internal
 peer PE connect-interface LoopBack0
 peer 2001:1::1 as-number 100
 peer 2001:1::1 group PE
 peer 2001:4::1 as-number 100
 peer 2001:4::1 group PE
 #
 l2vpn-family evpn
  reflector cluster-id 100
  undo policy vpn-target
  peer PE enable
  peer PE reflect-client
peer 2001:1::1 enable
  peer 2001:1::1 group PE
  peer 2001:4::1 enable
  peer 2001:4::1 group PE
#

检查命令:
display bgp evpn peer

4.5 SRv6基础配置

4.5.1 配置SRv6 SID

配置SRv6 SID,在PE上配置EVPN Type5路由携带的End.DT4/End.DT6 SID。

PE1配置:
#
segment-routing ipv6
 encapsulation source-address 2001:1::1
 locator PE1 ipv6-prefix 3001:1:: 80 static 15 args 16
  opcode ::10 end psp
  opcode ::12 end-x interface ether 3/0/0 nexthop 2001:12::2 psp
  opcode ::13 end-x interface ether 3/0/1 nexthop 2001:13::3 psp
  opcode ::4001 end-dt4 vpn-instance vpn1 evpn
  opcode ::6001 end-dt6 vpn-instance vpn1 evpn
  opcode ::7001 end-op
#配置SRv6 TE Policy关联Segment Routing IPv6下的一个Locator,
#并在关联Locator范围内指定SRv6 TE Policy的Binding SID
 srv6-te-policy locator PE1
#使能所有SRv6 TE Policy故障感知功能
 srv6-te-policy path verification enable
#

P2配置:
#
segment-routing ipv6
 encapsulation source-address 2001:2::1
 locator P2 ipv6-prefix 3001:2:: 80 static 15 args 16
  opcode ::10 end psp
  opcode ::21 end-x interface ether 3/0/1 nexthop 2001:12::1 psp
  opcode ::23 end-x interface ether 3/0/0 nexthop 2001:23::3 psp
  opcode ::24 end-x interface ether 3/0/2 nexthop 2001:24::4 psp
  opcode ::7001 end-op
 srv6-te-policy locator P2
#

P3配置:
#
segment-routing ipv6
 encapsulation source-address 2001:3::1
 locator P3 ipv6-prefix 3001:3:: 80 static 15 args 16
  opcode ::10 end psp
  opcode ::31 end-x interface ether 3/0/1 nexthop 2001:13::1 psp
  opcode ::32 end-x interface ether 3/0/0 nexthop 2001:23::2 psp
  opcode ::34 end-x interface ether 3/0/3 nexthop 2001:34::4 psp
  opcode ::7001 end-op
 srv6-te-policy locator P3
 #

PE4配置:
#
segment-routing ipv6
 encapsulation source-address 2001:4::1
 locator PE4 ipv6-prefix 3001:4:: 80 static 15 args 16
  opcode ::10 end psp
  opcode ::42 end-x interface ether 3/0/2 nexthop 2001:24::2 psp
  opcode ::43 end-x interface ether 3/0/3 nexthop 2001:34::3 psp
  opcode ::4001 end-dt4 vpn-instance vpn1 evpn
  opcode ::6001 end-dt6 vpn-instance vpn1 evpn
  opcode ::7001 end-op
 srv6-te-policy locator PE4
 srv6-te-policy path verification enable
#
 
检查命令:
display segment-routing ipv6 locator verbose 
display segment-routing ipv6 local-sid forwarding
display segment-routing ipv6 local-sid end forwarding 
(ProtocolType: STATIC就是这里分配的)

4.5.2 配置ISIS发送SID

ISIS通过LSP(TLV)发布Locator路由和SID信息。

segment-routing ipv6 locator PE1命令解释:

配置locator-name参数后,IS-IS支持从配置的locator-name下引入静态的End和End.X SID。另外IS-IS支持End和End.X SID动态分配能力。如果配置auto-sid-disable参数,表示取消动态End和End.X SID分配,完全从指定的Locator下引入静态的End和End.X SID属性。

PE1配置,其他类似,注意locator name不同:
isis 100
segment-routing ipv6 locator PE1

检查命令:
display segment-routing ipv6 local-sid end forwarding 
(ProtocolType: ISIS就是ISIS分配的)

dis isis lsdb is-name PE1 verbose
(查看ISIS LSP SID发布)

4.5.3 配置私网路由(即vpn路由)发送和迭代属性

使能私网路由上送EVPN协议时携带SID属性功能,通过bgp向远端PE发送end.dt4和end.dt6 sid。

segment-routing ipv6 locator PE1 evpn命令解释:

在EVPN L3VPN场景中,如果用户希望使用SRv6隧道承载业务流量,则需要配置segment-routing ipv6 locator evpn命令为将要发送的EVPN路由添加SID属性。对端BGP EVPN邻居在收到EVPN路由后,可以根据路由中的SID迭代SRv6隧道,从而生成转发表项。

segment-routing ipv6 traffic-engineer best-effort evpn命令解释:

vpn流量迭代到SRv6 TE Policy隧道,SRv6 TE Policy隧道故障时业务可以使用SRv6 BE隧道作为逃生路径。

PE1配置,PE4类似:
#
bgp 100
  #
 ipv4-family vpn-instance vpn1
   segment-routing ipv6 locator PE1 evpn
   segment-routing ipv6 traffic-engineer best-effort evpn  
#
 ipv6-family vpn-instance vpn1
  segment-routing ipv6 locator PE1 evpn
  segment-routing ipv6 traffic-engineer best-effort evpn
#

检查命令:
dis bgp evpn all routing-table prefix-route 0:172.20.1.0:24

4.5.4 配置BGP EVPN发送SRv6封装的EVPN路由

配置向EVPN邻居发布SRv6封装的EVPN路由。

peer RR advertise encap-type srv6命令解释:

缺省情况下,BGP EVPN邻居之间仅会相互发布MPLS封装的EVPN路由。但是在EVPN over SRv6场景中,BGP EVPN邻居之间需要发布SRv6封装的EVPN路由,此时需要配置peer advertise encap-type srv6命令。

PE1配置,PE4类似:
#
bgp 100
 #
 l2vpn-family evpn
peer RR advertise encap-type srv6
#

P2配置,P3类似:
#
bgp 100
 #
 l2vpn-family evpn
  peer PE advertise encap-type srv6
#

检查命令:
可以Wireshark抓取PE4发给RR的bgp update报文,查看bgp update里的evpn相关信息。

五、配置结果验证

5.1 检查ISIS邻居关系是否建立

以PE1为例:

[PE1]dis isis peer
 
                          Peer information for ISIS(100)
                         
  System Id     Interface          Circuit Id        State HoldTime Type     PRI
--------------------------------------------------------------------------------
P2*             Eth3/0/0           0000000006         Up   26s      L2       -- 
P3*             Eth3/0/1           0000000006         Up   22s      L2       -- 
 
Total Peer(s): 2
[PE1]

5.2 检查BGP EVPN邻居关系是否建立

以PE1为例:

[PE1]dis bgp evpn peer
 
 BGP local router ID : 192.168.1.1
 Local AS number : 100
 Total number of peers : 2                 Peers in established state : 2
 
  Peer                             V          AS  MsgRcvd  MsgSent  OutQ  Up/Down       State  PrefRcv
  2001:2::1                        4         100      322      318     0 04:32:44 Established        2
  2001:3::1                        4         100      321      320     0 04:33:46 Established        2
[PE1]

5.3 检查SRv6 本地Locator和SID信息

以PE1为例,其中:

静态SID由配置生成

动态end和end.x SID由ISIS协议动态生成,可以配置auto-sid-disable关闭。

[PE1]display segment-routing ipv6 locator verbose
 
                        Locator Configuration Table
                        ---------------------------
 
LocatorName   : PE1                                       LocatorID     : 1
IPv6Prefix    : 3001:1::                                  PrefixLength  : 80
Block         : --                                        BlockLength   : 0
NodeID        : --                                        NodeIdLength  : 0
ComprStaticLen: 0                                         StaticLength  : 15
ArgsLength    : 16                                        Reference     : 3
Algorithm     : 0                                         ComprDynLength: 0
AutoCSIDPoolID: 0                                        
AutoCSIDBegin : --
AutoCSIDEnd   : --
StaticCSIDBegin: --
StaticCSIDEnd : --
AutoSIDPoolID : 8193                                      DynLength     : 17
AutoSIDBegin  : 3001:1::8000:0
AutoSIDEnd    : 3001:1::FFFF:FFFF:0
StaticSIDBegin: 3001:1::1:0
StaticSIDEnd  : 3001:1::7FFF:0
GIB:LIB       : --
 
Total Locator(s): 1
[PE1]
 
 
[PE1]dis segment-routing ipv6 local-sid forwarding 
 
                    My Local-SID Forwarding Table
                    -------------------------------------
 
SID        : 3001:1::10:0/128                             FuncType : End
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::12:0/128                             FuncType : End.X
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::13:0/128                             FuncType : End.X
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::4001:0/128                           FuncType : End.DT4
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::6001:0/128                           FuncType : End.DT6
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
                
SID        : 3001:1::7001:0/128                           FuncType : End.OP
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::8003:0/128                           FuncType : End
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::8004:0/128                           FuncType : End
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::8005:0/128                           FuncType : End.X
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
SID        : 3001:1::8006:0/128                           FuncType : End.X
LocatorName: PE1                                          LocatorID: 1
SidCompress: NO
 
Total SID(s): 10
[PE1] 
 
[PE1]dis segment-routing ipv6 local-sid end forwarding 
 
                    My Local-SID End Forwarding Table
                    ---------------------------------
 
SID         : 3001:1::10:0/128                             FuncType    : End
Flavor      : PSP                                          SidCompress : NO
LocatorName : PE1                                          LocatorID   : 1
ProtocolType: STATIC                                       ProcessID   : --
UpdateTime  : 2023-09-19 01:33:05.136
 
SID         : 3001:1::8003:0/128                           FuncType    : End
Flavor      : NO-FLAVOR                                    SidCompress : NO
LocatorName : PE1                                          LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-19 06:17:13.116
 
SID         : 3001:1::8004:0/128                           FuncType    : End
Flavor      : PSP USP USD                                  SidCompress : NO
LocatorName : PE1                                          LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-19 06:17:13.116
 
Total SID(s): 3
[PE1] 
[PE1]

5.4 检查ISIS通过LSP发送的SID信息

ISIS通过LSP(TLV)会发送locator,end,end.x等信息,可以检查ISIS LSDB查看相关信息。

[PE1]dis isis lsdb
 
                        Database information for ISIS(100)
                        -----------------------------------
 
                          Level-2 Link State Database
 
LSPID                 Seq Num      Checksum      HoldTime      Length  ATT/P/OL
-------------------------------------------------------------------------------
PE1.00-00*            0x0000001d   0x6025        914           484     0/0/0   
P2.00-00              0x0000001c   0xc674        1036          595     0/0/0   
P3.00-00              0x00000020   0xebea        918           595     0/0/0   
PE4.00-00             0x0000001f   0xa568        928           484     0/0/0   
 
Total LSP(s): 4
    *(In TLV)-Leaking Route, *(By LSPID)-Self LSP, +-Self LSP(Extended),
           ATT-Attached, P-Partition, OL-Overload
 
 
[PE1]dis isis lsdb is-name PE1 verbose
 
                        Database information for ISIS(100)
                        -----------------------------------
                          
                          
                          Level-2 Link State Database
 
LSPID                  Seq Num    Checksum   HoldTime       Length   ATT/P/OL
-----------------------------------------------------------------------------
1921.6800.1001.00-00*  0x0000001d 0x6025     851            484      0/0/0   
 SOURCE       PE1.00 
 HOST NAME    PE1
 NLPID        IPV4
 NLPID        IPV6
 AREA ADDR    49.0001
 INTF ADDR    192.168.1.1
 INTF ADDR    192.168.12.1
 INTF ADDR    192.168.13.1
 INTF ADDR V6 2001:1::1
 INTF ADDR V6 2001:12::1
 INTF ADDR V6 2001:13::1
 Topology     Standard, IPV6
+NBR  ID      P3.00  COST: 10
+NBR  ID      P2.00  COST: 10
+MT NBR ID    P3.00  COST: 10            MT: 2 (IPV6)   <---发送end.x
   P2P-SRv6-X-Sid   3001:1::8005:0    B: 0  S: 0  P: 0  C: 0  Algorithm: 0  Weight: 0  
     Function: End.X      Flavor: NO-FLAVOR    
     Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --     
   P2P-SRv6-X-Sid   3001:1::13:0      B: 0  S: 0  P: 1  C: 0  Algorithm: 0  Weight: 0  
     Function: End.X      Flavor: PSP     
     Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --     
+MT NBR ID    P2.00  COST: 10            MT: 2 (IPV6)
   P2P-SRv6-X-Sid   3001:1::8006:0    B: 0  S: 0  P: 0  C: 0  Algorithm: 0  Weight: 0  
     Function: End.X      Flavor: NO-FLAVOR    
     Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --     
   P2P-SRv6-X-Sid   3001:1::12:0      B: 0  S: 0  P: 1  C: 0  Algorithm: 0  Weight: 0  
     Function: End.X      Flavor: PSP     
     Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --     
+IP-Extended  192.168.1.1     255.255.255.255  COST: 0         
+IP-Extended  192.168.12.0    255.255.255.0    COST: 10        
+IP-Extended  192.168.13.0    255.255.255.0    COST: 10        
 IPV6         2001:1::1/128                    COST: 0          MT: 2
 IPV6         3001:1::/80                      COST: 0          MT: 2  <--- locator路由
 IPV6         2001:12::/64                     COST: 10         MT: 2
 IPV6         2001:13::/64                     COST: 10         MT: 2
 Router Cap   192.168.1.1      D: 0  S: 0
   Segment Routing IPv6 Cap   O: 0   C: 0
   Segment Routing MSD   Max-SL: 10   Max-End-Pop: 11   Max-H-Ins: 10   Max-H-Encap: 10   Max-End-D: 11
   Segment Routing IPv6 Router ID    2001:1::1
 SRv6 Locator 3001:1::/80                      MT: 2 Metric: 0 D: 0 Algorithm: 0 <--- 发送end
   SRv6 End Sid   3001:1::10:0                     C: 0
      Function: End         Flavor: PSP      
      Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --      
   SRv6 End Sid   3001:1::8003:0                   C: 0
      Function: End         Flavor: NO-FLAVOR     
      Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --      
   SRv6 End Sid   3001:1::8004:0                   C: 0
      Function: End         Flavor: PSP USP USD      
      Block-Len: --  NodeID-Len: --  Func-Len: --  Args-Len: --      
 
Total LSP(s): 1
    *(In TLV)-Leaking Route, *(By LSPID)-Self LSP, +-Self LSP(Extended),
           ATT-Attached, P-Partition, OL-Overload
 
[PE1]

5.5 检查vpn路由在以evpn路由方式发送时携带的相关end.dt4或者end.dt6信息

在PE4检查172.20.1.0/24以及172:20:1::/64两条ipv4/ipv6 vpn路由对应的end.dt4, end.dt6信息:

[PE4]dis bgp evpn all routing-table                               
 
 
 Local AS number : 100
 
 BGP Local router ID is 192.168.4.1
 Status codes: * - valid, > - best, d - damped, x - best external, a - add path,
               h - history,  i - internal, s - suppressed, S - Stale
               Origin : i - IGP, e - EGP, ? - incomplete
 
 
 EVPN address family:
 Number of Ip Prefix Routes: 6
 Route Distinguisher: 100:1
       Network(EthTagId/IpPrefix/IpPrefixLen)                 NextHop
 *>i   0:172.10.1.0:24                                        2001:1::1                                    
 * i                                                          2001:1::1                                    
 *>i   0:[172:10:1::]:64                                      2001:1::1                                    
 * i                                                          2001:1::1                                    
 Route Distinguisher: 100:2
       Network(EthTagId/IpPrefix/IpPrefixLen)                 NextHop
 *>    0:172.20.1.0:24                                        0.0.0.0                                      
 *>    0:[172:20:1::]:64                                      ::                                           
[PE4]
[PE4]dis bgp evpn all routing-table prefix-route 0:172.20.1.0:24  
 
 
 BGP local router ID : 192.168.4.1
 Local AS number : 100
 Total routes of Route Distinguisher(100:2): 1
 BGP routing table entry information of 0:172.20.1.0:24:   <--- vpnv4 route
 Imported route.
 From: 0.0.0.0 (0.0.0.0) 
 Route Duration: 0d05h05m08s
 Direct Out-interface: Ethernet3/0/0
 Original nexthop: 172.20.1.1 
 Qos information : 0x0
 Ext-Community: RT <100 : 100>
 Prefix-sid: 3001:4::4001:0, Endpoint Behavior: 19   <--- end.dt4
 AS-path Nil, origin incomplete, MED 0, pref-val 0, valid, local, best, select, pre 255
 Route Type: 5 (Ip Prefix Route)
 Ethernet Tag ID: 0, IP Prefix/Len: 172.20.1.0/24, ESI: 0000.0000.0000.0000.0000, GW IP Address: 0.0.0.0
 Advertised to such 2 peers:
    2001:3::1
    2001:2::1
 
[PE4]dis bgp evpn all routing-table prefix-route 0:[172:20:1::]:64
 
 
 BGP local router ID : 192.168.4.1
 Local AS number : 100
 Total routes of Route Distinguisher(100:2): 1
 BGP routing table entry information of 0:[172:20:1::]:64: <--- vpnv6 route
 Imported route.
 From: :: (0.0.0.0) 
 Route Duration: 0d05h05m01s
 Direct Out-interface: Ethernet3/0/0
 Original nexthop: 172:20:1::1
 Qos information : 0x0
 Ext-Community: RT <100 : 100>
 Prefix-sid: 3001:4::6001:0, Endpoint Behavior: 18     <--- end.dt6
 AS-path Nil, origin incomplete, MED 0, pref-val 0, valid, local, best, select, pre 255
 Route Type: 5 (Ip Prefix Route)
 Ethernet Tag ID: 0, IPv6 Prefix/Len: 172:20:1::/64, ESI: 0000.0000.0000.0000.0000, GW IPv6 Address: ::
 Advertised to such 2 peers:
    2001:3::1
    2001:2::1
 
[PE4]

5.6 检查bgp evpn发送SRv6封装的evpn路由

抓取PE4发给RR的bgp update报文,并手工通过wireshark解析:

Frame 22: 394 bytes on wire (3152 bits), 394 bytes captured (3152 bits)
Ethernet II, Src: 38:05:00:11:03:02 (38:05:00:11:03:02), Dst: 38:06:00:11:03:02 (38:06:00:11:03:02)
Internet Protocol Version 6, Src: 2001:4::1, Dst: 2001:2::1
Transmission Control Protocol, Src Port: 58530, Dst Port: 179, Seq: 65, Ack: 413, Len: 320
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 148
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 125
    Path attributes
        Path Attribute - ORIGIN: INCOMPLETE
        Path Attribute - AS_PATH: empty
        Path Attribute - MULTI_EXIT_DISC: 0
        Path Attribute - LOCAL_PREF: 100
        Path Attribute - EXTENDED_COMMUNITIES
            Flags: 0xc0, Optional, Transitive, Complete
            Type Code: EXTENDED_COMMUNITIES (16)
            Length: 8
            Carried extended communities: (1 community)
                Route Target: 100:100 [Transitive 2-Octet AS-Specific]
        Path Attribute - BGP Prefix-SID
            Flags: 0xd0, Optional, Transitive, Extended-Length, Complete
                1... .... = Optional: Set
                .1.. .... = Transitive: Set
                ..0. .... = Partial: Not set
                ...1 .... = Extended-Length: Set
                .... 0000 = Unused: 0x0
            Type Code: BGP Prefix-SID (40)
            Length: 28
            SRv6 L3 Service
                Type: SRv6 L3 Service (5)
                Length: 25
                Reserved: 00
                SRv6 Service Sub-TLVs
                    SRv6 Service Sub-TLV - SRv6 SID Information
                        Type: SRv6 SID Information (1)
                        Length: 21
                        Reserved: 00
                        SRv6 SID Value: 3001:4::4001:0
                        SRv6 SID Flags: 0x00
                        SRv6 Endpoint Behavior: End.DT4 (0x0013)
                        Reserved: 00
                        SRv6 Service Data Sub-Sub-TLVs
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 57
            Address family identifier (AFI): Layer-2 VPN (25)
            Subsequent address family identifier (SAFI): EVPN (70)
            Next hop: 2001:4::1
                IPv6 Address: 2001:4::1
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                EVPN NLRI: IP Prefix route
                    Route Type: IP Prefix route (5)
                    Length: 34
                    Route Distinguisher: 0000006400000002 (100:2)
                    ESI: 00:00:00:00:00:00:00:00:00:00
                    Ethernet Tag ID: 0
                    IP prefix length: 24
                    IPv4 address: 172.20.1.0
                    IPv4 Gateway address: 0.0.0.0
                    MPLS Label Stack: 3 (bottom)
                        MPLS Label: 493 (bottom)
Border Gateway Protocol - UPDATE Message
    Marker: ffffffffffffffffffffffffffffffff
    Length: 172
    Type: UPDATE Message (2)
    Withdrawn Routes Length: 0
    Total Path Attribute Length: 149
    Path attributes
        Path Attribute - ORIGIN: INCOMPLETE
        Path Attribute - AS_PATH: empty
        Path Attribute - MULTI_EXIT_DISC: 0
        Path Attribute - LOCAL_PREF: 100
        Path Attribute - EXTENDED_COMMUNITIES
            Flags: 0xc0, Optional, Transitive, Complete
            Type Code: EXTENDED_COMMUNITIES (16)
            Length: 8
            Carried extended communities: (1 community)
                Route Target: 100:100 [Transitive 2-Octet AS-Specific]
        Path Attribute - BGP Prefix-SID
            Flags: 0xd0, Optional, Transitive, Extended-Length, Complete
            Type Code: BGP Prefix-SID (40)
            Length: 28
            SRv6 L3 Service
                Type: SRv6 L3 Service (5)
                Length: 25
                Reserved: 00
                SRv6 Service Sub-TLVs
                    SRv6 Service Sub-TLV - SRv6 SID Information
                        Type: SRv6 SID Information (1)
                        Length: 21
                        Reserved: 00
                        SRv6 SID Value: 3001:4::6001:0
                        SRv6 SID Flags: 0x00
                        SRv6 Endpoint Behavior: End.DT6 (0x0012)
                        Reserved: 00
                        SRv6 Service Data Sub-Sub-TLVs
        Path Attribute - MP_REACH_NLRI
            Flags: 0x90, Optional, Extended-Length, Non-transitive, Complete
            Type Code: MP_REACH_NLRI (14)
            Length: 81
            Address family identifier (AFI): Layer-2 VPN (25)
            Subsequent address family identifier (SAFI): EVPN (70)
            Next hop: 2001:4::1
                IPv6 Address: 2001:4::1
            Number of Subnetwork points of attachment (SNPA): 0
            Network Layer Reachability Information (NLRI)
                EVPN NLRI: IP Prefix route
                    Route Type: IP Prefix route (5)
                    Length: 58
                    Route Distinguisher: 0000006400000002 (100:2)
                    ESI: 00:00:00:00:00:00:00:00:00:00
                    Ethernet Tag ID: 0
                    IP prefix length: 64
                    IPv6 address: 172:20:1::
                    IPv6 Gateway address: ::
                    MPLS Label Stack: 3 (bottom)
                        MPLS Label: 493 (bottom)

六、SRv6 BE测试

6.1 测试拓扑

6.2 测试说明

前面的配置完成后,由于还没有进行SRv6 TE Policy配置,此时CE1 ping CE2的流量将迭代到SRv6 BE。

6.3 测试步骤

6.3.1 检查VPN路由

PE1检查vpn路由172.20.1.0/24和172:20:1::0/64已经迭代到SRv6 BE,并且直接以end.dt4/end.dt6为下一跳地址。

[PE1]dis ip routing-table vpn-instance vpn1  
Route Flags: R - relay, D - download to fib, T - to vpn-instance, B - black hole route
------------------------------------------------------------------------------
Routing Table : vpn1
         Destinations : 6        Routes : 6         
 
Destination/Mask    Proto   Pre  Cost        Flags NextHop                                  Interface
 
      127.0.0.0/8   Direct  0    0             D   127.0.0.1                                InLoopBack0
     172.10.1.0/24  Direct  0    0             D   172.10.1.1                               Ethernet3/0/2
     172.10.1.1/32  Direct  0    0             D   127.0.0.1                                Ethernet3/0/2
   172.10.1.255/32  Direct  0    0             D   127.0.0.1                                Ethernet3/0/2
     172.20.1.0/24  IBGP    255  0             RD  3001:4::4001:0                           SRv6 BE
255.255.255.255/32  Direct  0    0             D   127.0.0.1                                InLoopBack0
[PE1]
[PE1]dis ipv6 routing-table vpn-instance vpn1
Routing Table : vpn1
         Destinations : 4        Routes : 4         
 
Destination  : 172:10:1::                              PrefixLength : 64
NextHop      : 172:10:1::1                             Preference   : 0
Cost         : 0                                       Protocol     : Direct
RelayNextHop : ::                                      TunnelID     : 0x0
Interface    : Ethernet3/0/2                           Flags        : D
 
Destination  : 172:10:1::1                             PrefixLength : 128
NextHop      : ::1                                     Preference   : 0
Cost         : 0                                       Protocol     : Direct
RelayNextHop : ::                                      TunnelID     : 0x0
Interface    : Ethernet3/0/2                           Flags        : D
 
Destination  : 172:20:1::                              PrefixLength : 64
NextHop      : 3001:4::6001:0                          Preference   : 255
Cost         : 0                                       Protocol     : IBGP
RelayNextHop : 3001:4::6001:0                          TunnelID     : 0x0
Interface    : SRv6 BE                                 Flags        : RD
 
Destination  : FE80::                                  PrefixLength : 10
NextHop      : ::                                      Preference   : 0
Cost         : 0                                       Protocol     : Direct
RelayNextHop : ::                                      TunnelID     : 0x0
Interface    : NULL0                                   Flags        : DB
[PE1]

6.3.2 Ping测试

从CE1 ping CE2。

Welcome to use PC Simulators!
PC> 
PC> ping 172.20.1.10
PING 172.20.1.10 (172.20.1.10) 56(84) bytes of data.
64 bytes from 172.20.1.10: icmp_seq=1 ttl=62 time=86.5 ms
64 bytes from 172.20.1.10: icmp_seq=2 ttl=62 time=93.2 ms
64 bytes from 172.20.1.10: icmp_seq=3 ttl=62 time=83.9 ms
64 bytes from 172.20.1.10: icmp_seq=4 ttl=62 time=75.3 ms
64 bytes from 172.20.1.10: icmp_seq=5 ttl=62 time=102 ms
^C
--- 172.20.1.10 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4008ms
rtt min/avg/max/mdev = 75.325/88.263/102.382/9.085 ms
PC> ping 172:20:1::10
PING 172:20:1::10(172:20:1::10) 56 data bytes
64 bytes from 172:20:1::10: icmp_seq=1 ttl=62 time=123 ms
64 bytes from 172:20:1::10: icmp_seq=2 ttl=62 time=66.6 ms
64 bytes from 172:20:1::10: icmp_seq=3 ttl=62 time=147 ms
64 bytes from 172:20:1::10: icmp_seq=4 ttl=62 time=90.6 ms
^C
--- 172:20:1::10 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3019ms
rtt min/avg/max/mdev = 66.587/106.560/146.528/30.443 ms
PC> 

6.3.3 检查SRv6 BE封装格式

CE1 ping CE2时,在PE1,抓取PE1和P1、P2连接的端口报文,并手工通过wireshark解析。

这里以CE1 ping  CE2地址172.20.1.10进行抓包。

Frame 1: 138 bytes on wire (1104 bits), 138 bytes captured (1104 bits)
Ethernet II, Src: 38:05:00:11:03:00 (38:05:00:11:03:00), Dst: 38:07:00:11:03:01 (38:07:00:11:03:01)
Internet Protocol Version 6, Src: 2001:1::1, Dst: 3001:4::4001:0
    0110 .... = Version: 6
    .... 1111 1111 .... .... .... .... .... = Traffic Class: 0xff (DSCP: Unknown, ECN: CE)
    .... 0000 0000 0000 0000 0001 = Flow Label: 0x00001
    Payload Length: 84
    Next Header: IPIP (4)
    Hop Limit: 254
    Source Address: 2001:1::1
    Destination Address: 3001:4::4001:0  <---end.dt4地址作为目的地址
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.20.1.10
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
    Total Length: 84
    Identification: 0x0cbe (3262)
    010. .... = Flags: 0x2, Don't fragment
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 63
    Protocol: ICMP (1)
    Header Checksum: 0xd4b8 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 172.10.1.10
    Destination Address: 172.20.1.10
Internet Control Message Protocol

七、SRv6 TE Policy测试

7.1 测试拓扑

7.2 测试说明

PE1配置1条SRv6 TE Policy,命名为policy1

PE1接收EVPN路由172.20.1.0时对此路由进行染色,通过color引流的方式,将CE1 ping CE2的业务流量引入SRv6 TE Policy policy1

为简化测试,只在PE1配置SRv6 TE Policy,所以CE1 Ping CE2时,CE1去往CE2方向的流量会走SRv6 TE Policy,CE2返回CE1方向的流量走SRv6 BE

7.3 测试步骤

7.3.1 查看End SID信息

<PE1>dis segment-routing ipv6 local-sid end forwarding 
 
                    My Local-SID End Forwarding Table
                    ---------------------------------
 
SID         : 3001:1::10:0/128                             FuncType    : End
Flavor      : PSP                                          SidCompress : NO
LocatorName : PE1                                          LocatorID   : 1
ProtocolType: STATIC                                       ProcessID   : --
UpdateTime  : 2023-09-23 09:48:00.553
 
SID         : 3001:1::8021:0/128                           FuncType    : End
Flavor      : NO-FLAVOR                                    SidCompress : NO
LocatorName : PE1                                          LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 09:58:13.667
 
SID         : 3001:1::8022:0/128                           FuncType    : End
Flavor      : PSP USP USD                                  SidCompress : NO
LocatorName : PE1                                          LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 09:58:13.667
 
Total SID(s): 3
<PE1>
 
-------------
 
[P2]dis segment-routing ipv6 local-sid end forwarding 
 
                    My Local-SID End Forwarding Table
                    ---------------------------------
 
SID         : 3001:2::10:0/128                             FuncType    : End
Flavor      : PSP                                          SidCompress : NO
LocatorName : P2                                           LocatorID   : 1
ProtocolType: STATIC                                       ProcessID   : --
UpdateTime  : 2023-09-23 09:49:30.230
 
SID         : 3001:2::8024:0/128                           FuncType    : End
Flavor      : NO-FLAVOR                                    SidCompress : NO
LocatorName : P2                                           LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 09:59:43.963
 
SID         : 3001:2::8025:0/128                           FuncType    : End
Flavor      : PSP USP USD                                  SidCompress : NO
LocatorName : P2                                           LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 09:59:43.963
 
Total SID(s): 3
[P2]
 
------------------
 
[P3]dis segment-routing ipv6 local-sid end forwarding 
 
                    My Local-SID End Forwarding Table
                    ---------------------------------
 
SID         : 3001:3::10:0/128                             FuncType    : End
Flavor      : PSP                                          SidCompress : NO
LocatorName : P3                                           LocatorID   : 1
ProtocolType: STATIC                                       ProcessID   : --
UpdateTime  : 2023-09-23 09:47:17.136
 
SID         : 3001:3::8021:0/128                           FuncType    : End
Flavor      : NO-FLAVOR                                    SidCompress : NO
LocatorName : P3                                           LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 10:01:22.990
 
SID         : 3001:3::8022:0/128                           FuncType    : End
Flavor      : PSP USP USD                                  SidCompress : NO
LocatorName : P3                                           LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 10:01:22.990
 
Total SID(s): 3
[P3]
 
--------------
 
[PE4]dis segment-routing ipv6 local-sid end forwarding 
 
                    My Local-SID End Forwarding Table
                    ---------------------------------
 
SID         : 3001:4::10:0/128                             FuncType    : End
Flavor      : PSP                                          SidCompress : NO
LocatorName : PE4                                          LocatorID   : 1
ProtocolType: STATIC                                       ProcessID   : --
UpdateTime  : 2023-09-23 09:47:29.087
 
SID         : 3001:4::8000:0/128                           FuncType    : End
Flavor      : NO-FLAVOR                                    SidCompress : NO
LocatorName : PE4                                          LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 10:01:43.051
 
SID         : 3001:4::8001:0/128                           FuncType    : End
Flavor      : PSP USP USD                                  SidCompress : NO
LocatorName : PE4                                          LocatorID   : 1
ProtocolType: ISIS                                         ProcessID   : 100
UpdateTime  : 2023-09-23 10:01:43.051
 
Total SID(s): 3
[PE4]

7.3.2 配置SRv6 TE Policy

PE1配置:
#
segment-routing ipv6
 segment-list list1
  index 5 sid ipv6 3001:2::10:0
  index 10 sid ipv6 3001:4::10:0
 srv6-te policy policy1 endpoint 2001:4::1 color 101
  candidate-path preference 100
   segment-list list1
#

7.3.3 查看SRv6 TE Policy信息

[PE1]dis srv6-te policy policy-name policy1
PolicyName : policy1
Color                   : 101                            Endpoint             : 2001:4::1
TunnelId                : 3                              Binding SID          : -
TunnelType              : SRv6-TE Policy                 DelayTimerRemain     : -
Policy State            : Up                             State Change Time    : 2023-09-23 09:49:57
Admin State             : Up                             Traffic Statistics   : Disable
Backup Hot-Standby      : Disable                        BFD                  : Disable
Interface Index         : -                              Interface Name       : -
Interface State         : -                              Encapsulation Mode   : Insert
Candidate-path Count    : 1                             
 
 Candidate-path Preference : 100
 Path State             : Active                         Path Type            : Primary
 Protocol-Origin        : Configuration(30)              Originator           : 0, 0.0.0.0
 Discriminator          : 100                            Binding SID          : -
 GroupId                : 3                              Policy Name          : policy1
 Template ID            : 0                              Path Verification    : Enable
 DelayTimerRemain       : -                              Network Slice ID     : -
 Segment-List Count     : 1
  Segment-List          : list1
   Segment-List ID      : 3                              XcIndex              : 3  
   List State           : Up                             DelayTimerRemain     : -
   Verification State   : Up                             SuppressTimeRemain   : -
   PMTU                 : 9600                           Active PMTU          : 9600
   Weight               : 1                              BFD State            : -
   Network Slice ID     : -                             
   Binding SID          : -
   Reverse Binding SID  : -                             
   SID :
         3001:2::10:0                              
         3001:4::10:0                              
[PE1] 

7.3.4 配置Color引流

在PE1,对172.20.1.0/24 BGP 路由染色,color为101
#
#配置对BGP VPN路由进行染色
#
route-policy p1 permit node 10
 if-match ip-prefix 1
 apply extcommunity color 0:101
#
route-policy p1 permit node 20
#
ip ip-prefix 1 index 10 permit 172.20.1.0 24
#
bgp 100
 #
 l2vpn-family evpn
  peer RR route-policy p1 import
#
#配置隧道策略
#
tunnel-policy tnl-1
 tunnel select-seq ipv6 srv6-te-policy load-balance-number 1
#
#把隧道策略部署到VPN实例中
#
ip vpn-instance vpn1
 ipv4-family
  tnl-policy tnl-1 evpn
 ipv6-family
  tnl-policy tnl-1 evpn
#

7.3.5 查看BGP EVPN路由

[PE1]dis bgp evpn all routing-table 
 
 
 Local AS number : 100
 
 BGP Local router ID is 192.168.1.1
 Status codes: * - valid, > - best, d - damped, x - best external, a - add path,
               h - history,  i - internal, s - suppressed, S - Stale
               Origin : i - IGP, e - EGP, ? - incomplete
 
 
 EVPN address family:
 Number of Ip Prefix Routes: 8
 Route Distinguisher: 100:1
       Network(EthTagId/IpPrefix/IpPrefixLen)                 NextHop
 *>    0:172.10.1.0:24                                        0.0.0.0                                      
 *>    0:[172:10:1::]:64                                      ::                                           
 Route Distinguisher: 100:2
       Network(EthTagId/IpPrefix/IpPrefixLen)                 NextHop
 *>i   0:172.30.0.0:16                                        2001:4::1                                    
 * i                                                          2001:4::1                                    
 *>i   0:172.20.1.0:24                                        2001:4::1                                    
 * i                                                          2001:4::1                                    
 *>i   0:[172:20:1::]:64                                      2001:4::1                                    
 * i                                                          2001:4::1                                    
[PE1]
 
 
[PE1]dis bgp evpn all routing-table prefix-route 0:172.20.1.0:24
 
 
 BGP local router ID : 192.168.1.1
 Local AS number : 100
 Total routes of Route Distinguisher(100:2): 2
 BGP routing table entry information of 0:172.20.1.0:24:
 Label information (Received/Applied): 3/NULL
 From: 2001:2::1 (192.168.2.1) 
 Route Duration: 0d00h16m42s
 Relay IP Nexthop: FE80::3A06:FF:FE11:301
 Relay IP Out-Interface: Ethernet3/0/0
 Relay Tunnel Out-Interface: 
 Original nexthop: 2001:4::1
 Qos information : 0x0
 Ext-Community: RT <100 : 100>, Color <0 : 101>
 Prefix-sid: 3001:4::4001:0, Endpoint Behavior: 19
 AS-path Nil, origin incomplete, MED 0, localpref 100, pref-val 0, valid, internal, best, select, pre 255, IGP cost 20
 Originator: 192.168.4.1
 Cluster list: 0.0.0.100
 Route Type: 5 (Ip Prefix Route)
 Ethernet Tag ID: 0, IP Prefix/Len: 172.20.1.0/24, ESI: 0000.0000.0000.0000.0000, GW IP Address: 0.0.0.0
 Not advertised to any peer yet
 
 BGP routing table entry information of 0:172.20.1.0:24:
 Label information (Received/Applied): 3/NULL
 From: 2001:3::1 (192.168.3.1) 
 Route Duration: 0d00h18m04s
 Relay IP Nexthop: FE80::3A06:FF:FE11:301
 Relay IP Out-Interface: Ethernet3/0/0
 Relay Tunnel Out-Interface: 
 Original nexthop: 2001:4::1
 Qos information : 0x0
 Ext-Community: RT <100 : 100>, Color <0 : 101>
 Prefix-sid: 3001:4::4001:0, Endpoint Behavior: 19
 AS-path Nil, origin incomplete, MED 0, localpref 100, pref-val 0, valid, internal, pre 255, IGP cost 20, not preferred for peer address
 Originator: 192.168.4.1
 Cluster list: 0.0.0.100
 Route Type: 5 (Ip Prefix Route)
 Ethernet Tag ID: 0, IP Prefix/Len: 172.20.1.0/24, ESI: 0000.0000.0000.0000.0000, GW IP Address: 0.0.0.0
 Not advertised to any peer yet
 
[PE1] 

7.3.6 查看BGP VPN路由

[PE1]dis bgp vpnv4 vpn-instance vpn1 routing-table 
 
 BGP Local router ID is 192.168.1.1
 Status codes: * - valid, > - best, d - damped, x - best external, a - add path,
               h - history,  i - internal, s - suppressed, S - Stale
               Origin : i - IGP, e - EGP, ? - incomplete
 RPKI validation codes: V - valid, I - invalid, N - not-found
 
    
 VPN-Instance vpn1, Router ID 192.168.1.1:
 
 Total Number of Routes: 7
        Network            NextHop                       MED        LocPrf    PrefVal Path/Ogn
 
 *>     127.0.0.0/8        0.0.0.0                        0                     0       ?
 *>     172.10.1.0/24      0.0.0.0                        0                     0       ?
 *>     172.10.1.1/32      0.0.0.0                        0                     0       ?
 *>i    172.20.1.0/24      2001:4::1                      0          100        0       ?
 * i                       2001:4::1                      0          100        0       ?
 *>i    172.30.0.0/16      2001:4::1                      0          100        0       ?
 * i                       2001:4::1                      0          100        0       ?
[PE1]
 
 
[PE1]dis bgp vpnv4 vpn-instance vpn1 routing-table  172.20.1.0   
 
 BGP local router ID : 192.168.1.1
 Local AS number : 100
    
 VPN-Instance vpn1, Router ID 192.168.1.1:
 Paths:   2 available, 1 best, 1 select, 0 best-external, 0 add-path
 BGP routing table entry information of 172.20.1.0/24:
 Route Distinguisher: 100:2
 Remote-Cross route
 Evpn route: Type 5, ip-prefix
 Label information (Received/Applied): 3/NULL
 From: 2001:2::1 (192.168.2.1)
 Route Duration: 0d00h18m27s
 Relay IP Nexthop: FE80::3A06:FF:FE11:301
 Relay IP Out-Interface: Ethernet3/0/0
 Relay Tunnel Out-Interface: policy1(srv6tepolicy)
 Original nexthop: 2001:4::1
 Qos information : 0x0
 Ext-Community: RT <100 : 100>, Color <0 : 101>
 Prefix-sid: 3001:4::4001:0, Endpoint Behavior: 19
 AS-path Nil, origin incomplete, MED 0, localpref 100, pref-val 0, valid, internal, best, select, pre 255
 Originator: 192.168.4.1
 Cluster list: 0.0.0.100
 Not advertised to any peer yet
 
 BGP routing table entry information of 172.20.1.0/24:
 Route Distinguisher: 100:2
 Remote-Cross route
 Evpn route: Type 5, ip-prefix
 Label information (Received/Applied): 3/NULL
 From: 2001:3::1 (192.168.3.1)
 Route Duration: 0d00h18m27s
 Relay IP Nexthop: FE80::3A06:FF:FE11:301
 Relay IP Out-Interface: Ethernet3/0/0
 Relay Tunnel Out-Interface: policy1(srv6tepolicy)
 Original nexthop: 2001:4::1
 Qos information : 0x0
 Ext-Community: RT <100 : 100>, Color <0 : 101>
 Prefix-sid: 3001:4::4001:0, Endpoint Behavior: 19
 AS-path Nil, origin incomplete, MED 0, localpref 100, pref-val 0, valid, internal, pre 255, not preferred for peer address
 Originator: 192.168.4.1
 Cluster list: 0.0.0.100
 Not advertised to any peer yet
[PE1] 

7.3.7 查看VPN实例vpn1的路由

查看VPN实例IPv4路由表信息,可以看到私网路由已经成功迭代到SRv6 TE Policy。

[PE1]dis ip routing-table vpn-instance vpn1
Route Flags: R - relay, D - download to fib, T - to vpn-instance, B - black hole route
------------------------------------------------------------------------------
Routing Table : vpn1
         Destinations : 7        Routes : 7         
 
Destination/Mask    Proto   Pre  Cost        Flags NextHop                                  Interface
 
      127.0.0.0/8   Direct  0    0             D   127.0.0.1                                InLoopBack0
     172.10.1.0/24  Direct  0    0             D   172.10.1.1                               Ethernet3/0/2
     172.10.1.1/32  Direct  0    0             D   127.0.0.1                                Ethernet3/0/2
   172.10.1.255/32  Direct  0    0             D   127.0.0.1                                Ethernet3/0/2
     172.20.1.0/24  IBGP    255  0             RD  2001:4::1                                policy1
     172.30.0.0/16  IBGP    255  0             RD  2001:4::1                                SRv6-TE Flow Group
255.255.255.255/32  Direct  0    0             D   127.0.0.1                                InLoopBack0
[PE1]
 
 
[PE1]dis ip routing-table vpn-instance vpn1 172.20.1.0 verbose
Route Flags: R - relay, D - download to fib, T - to vpn-instance, B - black hole route
------------------------------------------------------------------------------
Routing Table : vpn1
Summary Count : 1
 
Destination: 172.20.1.0/24       
     Protocol: IBGP               Process ID: 0              
   Preference: 255                      Cost: 0              
      NextHop: 2001:4::1           Neighbour: 2001:2::1
        State: Active Adv Relied         Age: 00h20m50s           
          Tag: 0                    Priority: low            
        Label: NULL                  QoSInfo: 0x0           
   IndirectID: 0x10000B0            Instance:                                 
 RelayNextHop: ::                  Interface: policy1
     TunnelID: 0x000000003400000003    Flags: RD             
   RouteColor: 0                
[PE1]
[PE1]

7.3.8 Ping测试

CE1 ping CE2。


7.3.9 查看SRv6 TE Policy报文格式

CE1 ping CE2时,在PE1-Eth3/0/0、P2-Eth3/0/2抓取报文,并手工通过wireshark解析,查看SRH信息及IPv6目的地址的变化。

PE1-Eth3/0/0抓包:
 
Frame 2: 194 bytes on wire (1552 bits), 194 bytes captured (1552 bits)
Ethernet II, Src: 38:03:00:11:03:00 (38:03:00:11:03:00), Dst: 38:04:00:11:03:01 (38:04:00:11:03:01)
Internet Protocol Version 6, Src: ::, Dst: 3001:2::10:0
    0110 .... = Version: 6
    .... 1111 1111 .... .... .... .... .... = Traffic Class: 0xff (DSCP: Unknown, ECN: CE)
    .... 0000 0000 0000 0000 0001 = Flow Label: 0x00001
    Payload Length: 140
    Next Header: Routing Header for IPv6 (43)
    Hop Limit: 63
    Source Address: ::
    Destination Address: 3001:2::10:0
    Routing Header for IPv6 (Segment Routing)
        Next Header: IPIP (4)
        Length: 6
        [Length: 56 bytes]
        Type: Segment Routing (4)
        Segments Left: 2  <---------------待执行的SID数量
        Last Entry: 2
        Flags: 0x00
        Tag: 0000
        Address[0]: 3001:4::4001:0
        Address[1]: 3001:4::10:0
        Address[2]: 3001:2::10:0 <------------正在执行的SID
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.20.1.10
Internet Control Message Protocol
 
P2-Eth3/0/2抓包:
 
Frame 2: 194 bytes on wire (1552 bits), 194 bytes captured (1552 bits)
Ethernet II, Src: 38:04:00:11:03:02 (38:04:00:11:03:02), Dst: 38:05:00:11:03:02 (38:05:00:11:03:02)
Internet Protocol Version 6, Src: ::, Dst: 3001:4::10:0
    0110 .... = Version: 6
    .... 1111 1111 .... .... .... .... .... = Traffic Class: 0xff (DSCP: Unknown, ECN: CE)
    .... 0000 0000 0000 0000 0001 = Flow Label: 0x00001
    Payload Length: 140
    Next Header: Routing Header for IPv6 (43)
    Hop Limit: 62
    Source Address: ::
    Destination Address: 3001:4::10:0
    Routing Header for IPv6 (Segment Routing)
        Next Header: IPIP (4)
        Length: 6
        [Length: 56 bytes]
        Type: Segment Routing (4)
        Segments Left: 1   <---------------待执行的SID数量
        Last Entry: 2
        Flags: 0x00
        Tag: 0000
        Address[0]: 3001:4::4001:0
        Address[1]: 3001:4::10:0  <------------正在执行的SID
        Address[2]: 3001:2::10:0
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.20.1.10
Internet Control Message Protocol

八、SRv6 TE Flow Group测试

8.1 测试拓扑

8.2 测试说明

PE1配置2条SRv6 TE Policy,命名为policy10(color10)、policy11(color11)

PE4 vpn1配置测试loopback100、loopback101,用于ping测试

PE4发布vpn路由172.30.0.0/16

PE1配置Mapping policy,用于DSCP引流

PE1接收EVPN路由172.30.0.0/16时对此路由进行染色,通过DSCP引流的方式,将业务流量引入动态创建的SRv6 TE Flow Group

由于目前模拟路由器不支持DSCP标记,在CE1 ping 测试时,设置相应DSCP(支持linux ping)

CE1 ping相同网段(172.30.0.0/16)的不同地址,PE1根据DSCP,选择不同的SRv6 TE Policy进行传输

8.3 测试步骤

8.3.1 配置SRv6 TE Policy

PE1配置:
segment-routing ipv6
 segment-list list10
  index 5 sid ipv6 3001:2::10:0
  index 10 sid ipv6 3001:4::10:0
 segment-list list11
  index 5 sid ipv6 3001:3::10:0
  index 10 sid ipv6 3001:4::10:0
 srv6-te policy policy10 endpoint 2001:4::1 color 10
  binding-sid 3001:1::90:0
  candidate-path preference 100
   segment-list list10
 srv6-te policy policy11 endpoint 2001:4::1 color 11
  binding-sid 3001:1::91:0
  candidate-path preference 100
   segment-list list11
#

8.3.2 查看SRv6 TE Policy信息

[PE1]dis srv6-te policy
PolicyName : policy10
Color                   : 10                             Endpoint             : 2001:4::1
TunnelId                : 1                              Binding SID          : 3001:1::90:0(Insert)
TunnelType              : SRv6-TE Policy                 DelayTimerRemain     : -
Policy State            : Up                             State Change Time    : 2023-09-27 04:56:39
Admin State             : Up                             Traffic Statistics   : Disable
Backup Hot-Standby      : Disable                        BFD                  : Disable
Interface Index         : -                              Interface Name       : -
Interface State         : -                              Encapsulation Mode   : Insert
Candidate-path Count    : 1                             
 
 Candidate-path Preference : 100
 Path State             : Active                         Path Type            : Primary
 Protocol-Origin        : Configuration(30)              Originator           : 0, 0.0.0.0
 Discriminator          : 100                            Binding SID          : 3001:1::90:0
 GroupId                : 1                              Policy Name          : policy10
 Template ID            : 0                              Path Verification    : Enable
 DelayTimerRemain       : -                              Network Slice ID     : -
 Segment-List Count     : 1
  Segment-List          : list10
   Segment-List ID      : 1                              XcIndex              : 2  
   List State           : Up                             DelayTimerRemain     : -
   Verification State   : Up                             SuppressTimeRemain   : -
   PMTU                 : 9600                           Active PMTU          : 9600
   Weight               : 1                              BFD State            : -
   Network Slice ID     : -                             
   Binding SID          : -
   Reverse Binding SID  : -                             
   SID :
         3001:2::10:0                              
         3001:4::10:0                              
 
PolicyName : policy11
Color                   : 11                             Endpoint             : 2001:4::1
TunnelId                : 2                              Binding SID          : 3001:1::91:0(Insert)
TunnelType              : SRv6-TE Policy                 DelayTimerRemain     : -
Policy State            : Up                             State Change Time    : 2023-09-27 04:54:41
Admin State             : Up                             Traffic Statistics   : Disable
Backup Hot-Standby      : Disable                        BFD                  : Disable
Interface Index         : -                              Interface Name       : -
Interface State         : -                              Encapsulation Mode   : Insert
Candidate-path Count    : 1                             
 
 Candidate-path Preference : 100
 Path State             : Active                         Path Type            : Primary
 Protocol-Origin        : Configuration(30)              Originator           : 0, 0.0.0.0
 Discriminator          : 100                            Binding SID          : 3001:1::91:0
 GroupId                : 2                              Policy Name          : policy11
 Template ID            : 0                              Path Verification    : Enable
 DelayTimerRemain       : -                              Network Slice ID     : -
 Segment-List Count     : 1
  Segment-List          : list11
   Segment-List ID      : 2                              XcIndex              : 1  
   List State           : Up                             DelayTimerRemain     : -
   Verification State   : Up                             SuppressTimeRemain   : -
   PMTU                 : 9600                           Active PMTU          : 9600
   Weight               : 1                              BFD State            : -
   Network Slice ID     : -                             
   Binding SID          : -
   Reverse Binding SID  : -                             
   SID :
         3001:3::10:0                              
         3001:4::10:0   

8.3.3 配置测试Loopback地址

PE4创建两个loopback地址,分别配置测试网段172.30.1.0/24, 172.30.2.0/24,但对外只发布172.30.0.0/16的路由。

PE4:
#
interface LoopBack100
 ip binding vpn-instance vpn1
 ip address 172.30.1.1 255.255.255.0
#
interface LoopBack101
 ip binding vpn-instance vpn1
 ip address 172.30.2.1 255.255.255.0
#

8.3.4 发布VPN路由

PE4上,bgp ipv4 vpn实例vpn1在引入直连路由时,不引入loopback100/loopback101直连路由,通过引入静态路由方式,引入172.30.0.0/16,并发给PE1,这样做的目的是:PE1的vpn1都通过172.30.0.0/16访问172.30.1.1和172.30.2.1,并通过dscp对应不同的小color,迭代到不同的policy转发路径。

PE4配置
#
bgp 100
 #
 ipv4-family vpn-instance vpn1
  import-route direct route-policy p1
  import-route static
 #
route-policy p1 permit node 10
 if-match interface Ethernet3/0/0 (只允许局域端口的直连路由)
#
ip route-static vpn-instance vpn1 172.30.0.0 255.255.0.0 NULL0
#

执行完以上步骤后,在PE1用以下命令查看,172.30.0.0/16未染色之前,走SRv6 BE路径:

[PE1]dis ip routing-table vpn-instance vpn1

8.3.5 配置SRv6 Mapping Policy

8.3.5.1 功能说明

通过配置SRv6 Mapping Policy,设备利用业务路由(即VPN路由或私网路由)Color属性去匹配相同Color的SRv6 Mapping Policy,如果SRv6 Mapping Policy存在,则设备动态生成一个SRv6 TE Flow Group,供业务转发使用。该SRv6 TE Flow Group里存在多个Color属性不同,但EndPoint相同的SRv6 TE Policy。

8.3.5.2 DSCP说明

源IP地址

目的IP地址

模拟数据流

DSCP设置

Color

172.10.1.10

172.30.1.1

数据流1

AF31(DSCP:26,011010)

10

172.10.1.10

172.30.2.1

数据流2

AF21(DSCP:18,010010)

11

8.3.5.3 配置Mapping policy

PE1配置:
#
segment-routing ipv6
 mapping-policy p1 color 1000
  match-type dscp
   index 100 dscp ipv4 26 match srv6-te-policy color 10
   index 200 dscp ipv4 18 match srv6-te-policy color 11
#

8.3.6 配置DSCP引流

route-policy p1 permit node 20
 if-match ip-prefix 2
 apply extcommunity color 0:1000
#
route-policy p1 permit node 30
#
ip ip-prefix 1 index 10 permit 172.20.1.0 24
ip ip-prefix 2 index 10 permit 172.30.0.0 16
#
#
tunnel-policy tnl-1
 tunnel select-seq ipv6 srv6-te-policy load-balance-number 1
#
tunnel-policy tnl-2
 tunnel select-seq ipv6 srv6-te-flow-group srv6-te-policy load-balance-number 1 unmix
#
#
ip vpn-instance vpn1
 ipv4-family
  tnl-policy tnl-2 evpn
#

8.3.7 查看SRv6 TE Flow Group信息

[PE1]dis srv6-te flow-group
                                SRv6-TE Flow Group Information
----------------------------------------------------------------------------------------------
Group Name            :
Color                 : 1000                          Endpoint            : 2001:4::1
Group Tunnel ID       : 4                             Group Tunnel Type   : SRv6-TE Flow Group
Group Tunnel State    : Up                            State Change Time   : 2023-09-27 04:54:43
Interface Index       : -                             Interface Name      : -
Interface State       : -                             
Delay Timer Remain    : -                             UP/ALL Num          : 2/2
 
 Index                : 100                           AfType              : IPv4
 DSCP                 : 26
 Match Tunnel         : SRv6-TE Policy                State               : Up
 Color                : 10                            Tunnel Id           : 1
 
 Index                : 200                           AfType              : IPv4
 DSCP                 : 18
 Match Tunnel         : SRv6-TE Policy                State               : Up
 Color                : 11                            Tunnel Id           : 2
[PE1]

8.3.8 查看VPN实例vpn1的路由

查看VPN实例IPv4路由表信息,可以看到私网路由已经成功迭代到SRv6 TE Flow Group。

[PE1]dis ip routing-table vpn-instance vpn1
Route Flags: R - relay, D - download to fib, T - to vpn-instance, B - black hole route
------------------------------------------------------------------------------
Routing Table : vpn1
         Destinations : 7        Routes : 7         
 
Destination/Mask    Proto   Pre  Cost        Flags NextHop                                  Interface
 
      127.0.0.0/8   Direct  0    0             D   127.0.0.1                                InLoopBack0
     172.10.1.0/24  Direct  0    0             D   172.10.1.1                               Ethernet3/0/2
     172.10.1.1/32  Direct  0    0             D   127.0.0.1                                Ethernet3/0/2
   172.10.1.255/32  Direct  0    0             D   127.0.0.1                                Ethernet3/0/2
     172.20.1.0/24  IBGP    255  0             RD  2001:4::1                                policy1
     172.30.0.0/16  IBGP    255  0             RD  2001:4::1                                SRv6-TE Flow Group
255.255.255.255/32  Direct  0    0             D   127.0.0.1                                InLoopBack0
[PE1]

8.3.9 Ping测试

8.3.9.1 测试说明

由于目前ENSP Pro的NE路由器版本不支持QoS,所以在PC侧设置DSCP值。

ping -Q 104 (设置DSCP为26,即AF31)

ping -Q 72 (设置DSCP为18(010010),即AF21, 计算方式:IP ToS为1个字节,共8个bit, 7到2bit位为:010010,补充1-0bit位:00,01001000换算10进制为72)

8.3.9.2 查看DSCP是否正确设置

CE1 ping测试时,在PE1-Eth3/0/2抓包,查看DSCP是否正确设置。

ping -Q 104 172.30.1.1 (设置DSCP为26(AF31)),抓包结果符合预期:

Frame 1: 98 bytes on wire (784 bits), 98 bytes captured (784 bits)
Ethernet II, Src: b2:e0:61:ae:0e:be (b2:e0:61:ae:0e:be), Dst: 38:03:00:11:03:02 (38:03:00:11:03:02)
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.30.1.1
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x68 (DSCP: AF31, ECN: Not-ECT)
        0110 10.. = Differentiated Services Codepoint: Assured Forwarding 31 (26)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 84
    Identification: 0xe123 (57635)
    010. .... = Flags: 0x2, Don't fragment
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 64
    Protocol: ICMP (1)
    Header Checksum: 0xfee9 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 172.10.1.10
    Destination Address: 172.30.1.1
Internet Control Message Protocol

ping -Q 72 172.30.2.1 (设置DSCP为18(AF21)),抓包结果符合预期:

Frame 1: 98 bytes on wire (784 bits), 98 bytes captured (784 bits)
Ethernet II, Src: b2:e0:61:ae:0e:be (b2:e0:61:ae:0e:be), Dst: 38:03:00:11:03:02 (38:03:00:11:03:02)
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.30.2.1
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x48 (DSCP: AF21, ECN: Not-ECT)
        0100 10.. = Differentiated Services Codepoint: Assured Forwarding 21 (18)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 84
    Identification: 0x21f7 (8695)
    010. .... = Flags: 0x2, Don't fragment
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 64
    Protocol: ICMP (1)
    Header Checksum: 0xbd36 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 172.10.1.10
    Destination Address: 172.30.2.1
Internet Control Message Protocol

8.3.9.3 CE1 ping -Q 104 172.30.1.1

在PE1-3/0/0和3/0/1同时抓包,结果能正常ping通,3/0/0能抓到发出的包,3/0/1没有抓到发出的包。经过抓包分析,此数据流通过SRv6 TE Policy policy10传输,符合预期:

Frame 2: 194 bytes on wire (1552 bits), 194 bytes captured (1552 bits)
Ethernet II, Src: 38:03:00:11:03:00 (38:03:00:11:03:00), Dst: 38:06:00:11:03:01 (38:06:00:11:03:01)
Internet Protocol Version 6, Src: ::, Dst: 3001:2::10:0
    0110 .... = Version: 6
    .... 1111 1111 .... .... .... .... .... = Traffic Class: 0xff (DSCP: Unknown, ECN: CE)
        .... 1111 11.. .... .... .... .... .... = Differentiated Services Codepoint: Unknown (63)
        .... .... ..11 .... .... .... .... .... = Explicit Congestion Notification: Congestion Experienced (3)
    .... 0000 0000 0000 0000 0001 = Flow Label: 0x00001
    Payload Length: 140
    Next Header: Routing Header for IPv6 (43)
    Hop Limit: 63
    Source Address: ::
    Destination Address: 3001:2::10:0
    Routing Header for IPv6 (Segment Routing)
        Next Header: IPIP (4)
        Length: 6
        [Length: 56 bytes]
        Type: Segment Routing (4)
        Segments Left: 2
        Last Entry: 2
        Flags: 0x00
        Tag: 0000
        Address[0]: 3001:4::4001:0
        Address[1]: 3001:4::10:0
        Address[2]: 3001:2::10:0
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.30.1.1
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x68 (DSCP: AF31, ECN: Not-ECT)
        0110 10.. = Differentiated Services Codepoint: Assured Forwarding 31 (26)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 84
    Identification: 0xac39 (44089)
    010. .... = Flags: 0x2, Don't fragment
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 63
    Protocol: ICMP (1)
    Header Checksum: 0x34d4 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 172.10.1.10
    Destination Address: 172.30.1.1
Internet Control Message Protocol

8.3.9.4 CE1 ping -Q 72 172.30.2.1

在PE1-3/0/0和3/0/1同时抓包,结果能正常ping通,3/0/1能抓到发出的包,3/0/0没有抓到发出的包。经过抓包分析,此数据流通过SRv6 TE Policy policy11传输,符合预期:

Frame 2: 194 bytes on wire (1552 bits), 194 bytes captured (1552 bits)
Ethernet II, Src: 38:03:00:11:03:01 (38:03:00:11:03:01), Dst: 38:02:00:11:03:01 (38:02:00:11:03:01)
    Destination: 38:02:00:11:03:01 (38:02:00:11:03:01)
    Source: 38:03:00:11:03:01 (38:03:00:11:03:01)
    Type: IPv6 (0x86dd)
Internet Protocol Version 6, Src: ::, Dst: 3001:3::10:0
    0110 .... = Version: 6
    .... 1111 1111 .... .... .... .... .... = Traffic Class: 0xff (DSCP: Unknown, ECN: CE)
        .... 1111 11.. .... .... .... .... .... = Differentiated Services Codepoint: Unknown (63)
        .... .... ..11 .... .... .... .... .... = Explicit Congestion Notification: Congestion Experienced (3)
    .... 0000 0000 0000 0000 0001 = Flow Label: 0x00001
    Payload Length: 140
    Next Header: Routing Header for IPv6 (43)
    Hop Limit: 63
    Source Address: ::
    Destination Address: 3001:3::10:0
    Routing Header for IPv6 (Segment Routing)
        Next Header: IPIP (4)
        Length: 6
        [Length: 56 bytes]
        Type: Segment Routing (4)
        Segments Left: 2
        Last Entry: 2
        Flags: 0x00
        Tag: 0000
        Address[0]: 3001:4::4001:0
        Address[1]: 3001:4::10:0
        Address[2]: 3001:3::10:0
Internet Protocol Version 4, Src: 172.10.1.10, Dst: 172.30.2.1
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x48 (DSCP: AF21, ECN: Not-ECT)
        0100 10.. = Differentiated Services Codepoint: Assured Forwarding 21 (18)
        .... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
    Total Length: 84
    Identification: 0x3819 (14361)
    010. .... = Flags: 0x2, Don't fragment
    ...0 0000 0000 0000 = Fragment Offset: 0
    Time to Live: 63
    Protocol: ICMP (1)
    Header Checksum: 0xa814 [validation disabled]
    [Header checksum status: Unverified]
    Source Address: 172.10.1.10
    Destination Address: 172.30.2.1
Internet Control Message Protocol

--本文完,文章转至:https://blog.csdn.net/googleccsdn/article/details/133377356


Comment