Problem Statement
The local-mac-address default setting in the OBP is:
local-mac-address? false
Therefore all network interfaces in a node have the same MAC addresses.
If only one public interface is being used with this configuration, no
problems are encountered in a SunCluster 2.x configuration. The same is
true if using public network management.
In the following network configuration examples however, it will be
necessary to have various MAC addresses on all installed network inter-
faces in a node.
The following examples apply for 2 or more interfaces.
*************************************************************************
Example 1 - all 3 network interfaces with the same MAC address
on one ethernet segment :
All interfaces are connected to the same network segment. If the node gets
a request with its MAC address, all network interfaces in the node will
answer. In this example the request will be answered 3 times.
Disadvantage: high network load and bad network performance.
-----------
| Node |
| |
-----------
| | |
| | |
| | |
SegmA---------------------ClientsA
*************************************************************************
Example 2 - all 3 network interfaces with the same MAC address
on 3 bridged ethernet segments :
All interfaces are connected in several segments, but the networks are
aligned with a bridge. This will cause the same problem as in Example 1
because the bridge gives the information to the other segment. If the
node gets a request with its MAC address, all network interfaces in the
node will answer. In this example the request will be answered 3 times.
This example applies also for the behavior with three switches.
Disadvantage: high network load and bad network performance.
-----------
| Node |
| |
-----------
| | |
| | |
| | |
SegmA-----------|--|---------------|----------ClientsA
| | ----------
| | | Bridge |
| | ----------
SegmB-----------|--|---------------|----------ClientsB
| ----------
| | Bridge |
| ----------
SegmC--------------|-------|------------------ClientsC
*************************************************************************
Example 3 - all 3 network interfaces with the same MAC address
on one ethernet (layer 2) switch
All interfaces are connected to one ethernet switch (for example, the
SunSwitch). Now the behavior is influenced by the capabilities of the
switch. The switch builds up a table, where ports and MAC addresses are
combined. Only one port with one MAC address can be on top of the switch
table, so ALL packets are switched to ONE of the interfaces, only. The
interface changes if the node sends packets from another public interface,
but only one interface is active at a time.
Disadvantage: overall network performance is as fast as one of the interfaces.
-----------
| Node |
| |
-----------
| | |
| | |
| | |
-----------
| Switch |
| |
-----------
| | |
| | |
| | |
Clients A B C
Resolution
Set the entry in OBP as follows:
local-mac-address? true
With this setting, all network interfaces with their own MACaddress
onboard will get it from the hardware.
Be aware that the SunCluster 2.x will store the MAC address from the
private network in the /etc/opt/SUNWcluster/conf/.cdb file.
Supported private networks in the Sun Cluster 2.x are (Feb 2000):
SCI (Scalable Coherent Interface)
no problems because SBus SCI does not use a MAC address.
hme (Sun FastEthernet)
no problems because SBus hme always uses the MAC address
from the NVRAM.
qfe (Quad Fast Ethernet)
have a MAC address for each port onboard. If you have a
qfe interface in the private network, you should use the
mac-address from the NVRAM.
Three possibilities for configurations for qfe, or other network
interfaces in the future, which will have a mac-address onboard:
1) Install the Sun Cluster when the local-mac-address is set to false,
and give the scinstall script the mac-address from the NVRAM.
After the Sun Cluster installation, set the local-mac-address to
true and reboot the system. If you start the SunCluster now, the
private network interfaces will get the mac-addresses which are
stored in the /etc/opt/SUNWcluster/conf/.cdb file.
And all other qfe interfaces will have a unique mac-address.
Also if you swap the hardware, you will not have any problems!
2) If you have installed the Sun Cluster with local-mac-address true,
you can change the mac-address in the
/etc/opt/SUNWcluster/conf/.cdb file with the command:
scconf -N
If you use the command you must run it on all nodes in the Cluster
because each node only modifies its own .cdb file.
The private network interfaces will be plumbed with the mac-address
from the .cdb file.
3) If you like to configure the mac-address within the boot, you will
do this in
/etc/rcS.d/S30rootusr.sh (<= SunOS 5.7)
/etc/rcS.d/S30network.sh (>= SunOS 5.8)
with the command
ifconfig ether 8:0:20:83:fe:75
For this purpose you have to inactivate the ability to unload the
respective interface driver. An unload would reset the changes.
Add a forceload line with your driver to /etc/systems, e.g.:
forceload: drv/hme
forceload: drv/qfe
This possibility can be used to keep the same MAC address on all interfaces
within a nafo group.
Do not use the /opt/SUNWpnm/bin/staticroutes/staticroutes.nafo* file to set
MAC addresses, this would trigger BugID 4281521 .
What about pnm when the local-mac-address is set to true?
---------------------------------------------------------
If one interface fails in the nafo group, the other would get its
own mac-address. Therefore, the interface taking over will send a new
ARP Request which the Clients must learn (gratious ARP). This has
been tested with Sun clients and Windows 98 clients - both could
learn the new ARP request immediately.
Possibility of problems in configurations with routers.
Configuration Example:
-----------
| Node |
| |
-----------
| | |
| | |
| | |
NetA-----------|--|---------------|----------ClientsA
| | ----------
| | | Router |
| | ----------
NetB-----------|--|-------|-------|----------ClientsB
| ----------
| | Router |
| ----------
NetC--------------|-------|------------------ClientsC
The function is dependent on the ARP configuration of the routers.
Some routers don't learn new ARP request because they reset the
ARP table
e.g. 4 hours. Therefore the client is unable to connect
to the logical host after a pnm failover until the router has done
its reset. This will also happen when a logical host does a failover
from one node to the other because both have various MAC addresses,
and are independent from the local-mac-address.
You have to setup your routers for pnm and logical hosts.
For more information please refer to
INFODOC ID 18627: Debugging pnmd & nafo group problems
Additional comment:
------------------
If you have set the MAC address with the above ifconfig command, it
can only be cleared by the following:
- reboot the node
- use the ifconfig command again.
It can not be cleared with unplumb and plumb.
