0 0 Share PDF

Windows overlay network issue with Linux MTU

Issue

HRM seems to break Websocket communications to a service when websocket messages exceed +/- 1KB.

Prerequisites

  • UCP installed on Linux with HRM enabled.
  • A Windows worker node behind HyperV.
  • A service running on Windows attached to an overlay network router with HRM.

Root Cause

The Windows platform does not handle fragmented VXLAN packets correctly. In an overlay network IP, datagram are encapsulated in VXLAN which require 50 more bytes. On Linux when you create an overlay network it automatically set the mtu to 1450, but in Windows it stays at 1500, which can cause fragmentation.

Resolution

Because VXLAN add 50 bytes, lower the overlay MTU to make sure it doesn't reach the 1500.

For example:

networks:
  name:
    driver: overlay
    driver_opts:
      com.docker.network.driver.mtu: 1400