26 June 93 Sean Fagan called; PA is offline. Symptom from my side: no route to PA. We rebooted the 23.253 router and within 30 sec (RIP) it came back online. Suggested that next time we login, capture the (bad) route table and compare, contact Livingston. Here is the traceroute, during which the router restarted and RIP makde a route: traceroute 140.174.23.13 traceroute to 140.174.23.13 (140.174.23.13), 30 hops max, 40 byte packets 1 140.174.74.1 (140.174.74.1) 200 ms 210 ms 210 ms 2 gw.toad.com (140.174.2.23) 210 ms 190 ms 180 ms 3 gw-pa-sf.cygnus.com (140.174.5.2) 200 ms 220 ms 190 ms 4 gw-mv-pa.cygnus.com (140.174.9.1) 190 ms 220 ms 200 ms 5 CISCO.BA.TIS.COM (192.70.178.4) 220 ms 210 ms 220 ms 6 gw-mv-pa.cygnus.com (140.174.9.1) 230 ms 190 ms 210 ms 7 CISCO.BA.TIS.COM (192.70.178.4) 230 ms 230 ms 210 ms 8 gw-mv-pa.cygnus.com (140.174.9.1) 240 ms 210 ms 230 ms 9 CISCO.BA.TIS.COM (192.70.178.4) 220 ms 220 ms 210 ms 10 gw-mv-pa.cygnus.com (140.174.9.1) 210 ms 230 ms 200 ms 11 CISCO.BA.TIS.COM (192.70.178.4) 230 ms 220 ms 220 ms 12 gw-mv-pa.cygnus.com (140.174.9.1) 210 ms nos1.cygnus.com (140.174.23.13) 330 ms 240 ms Here is the good table from 23.13, after the reboot of 23.253: Destination Gateway Flag Met Interface --------------------- -------------------- ---- --- --------- 0.0.0.0 140.174.23.253 NS 1 ether0 140.174.85.98 140.174.85.98 HL 1 ptp7 140.174.85.2 140.174.85.2 HL 1 ptp1 140.174.85.226 140.174.85.226 HL 1 ptp5 140.174.85.130 140.174.85.130 HL 1 ptp3 140.174.85.66 140.174.85.66 HL 1 ptp2 140.174.85.194 140.174.85.194 HL 1 ptp4 192.9.201.70 140.174.85.2 HS 1 ptp1 192.9.201.61 140.174.85.2 HS 1 ptp1 140.174.86.0 140.174.85.2 NS 1 ptp1 140.174.87.0 140.174.85.130 NS 1 ptp3 140.174.23.0 140.174.23.13 NL 1 ether0 140.174.88.0 140.174.85.66 NS 1 ptp2 140.174.89.0 140.174.85.194 NS 1 ptp4 140.174.90.0 140.174.85.226 NS 1 ptp5 140.174.92.0 140.174.85.98 NS 1 ptp7 140.174.91.0 140.174.85.34 NS 1 Unknown From 23.253: Destination Gateway Flag Met Interface --------------------- -------------------- ---- --- --------- 0.0.0.0 140.174.24.2 NS 1 ptp1 140.174.8.192 140.174.24.2 HD 5 ptp1 140.174.8.128 140.174.24.2 HD 5 ptp1 140.174.8.64 140.174.24.2 HD 5 ptp1 140.174.24.2 140.174.24.2 HL 1 ptp1 140.174.1.0 140.174.24.2 ND 3 ptp1 140.174.66.0 140.174.24.2 ND 5 ptp1 140.174.2.0 140.174.24.2 ND 4 ptp1 140.174.69.0 140.174.24.2 ND 5 ptp1 140.174.5.0 140.174.24.2 ND 3 ptp1 140.174.70.0 140.174.24.2 ND 5 ptp1 140.174.71.0 140.174.24.2 ND 5 ptp1 140.174.7.0 140.174.24.2 ND 5 ptp1 140.174.72.0 140.174.24.2 ND 5 ptp1 140.174.8.0 140.174.24.2 ND 5 ptp1 140.174.73.0 140.174.24.2 ND 5 ptp1 140.174.9.0 140.174.24.2 ND 2 ptp1 140.174.74.0 140.174.24.2 ND 5 ptp1 140.174.42.0 140.174.24.2 ND 5 ptp1 140.174.75.0 140.174.24.2 ND 5 ptp1 -- More (21) -- From sef@cygnus.com Sun Jun 27 15:39:16 1993 Received: from cygnus.com by fido.wps.com (5.67/9999.1) id AA00498; Sun, 27 Jun 93 15:39:13 -0700 Received: by cygnus.com (4.1/SMI-4.1) id AA14792; Sun, 27 Jun 93 15:39:16 PDT Date: Sun, 27 Jun 93 15:39:16 PDT From: sef@cygnus.com (Sean Eric Fagan) Message-Id: <9306272239.AA14792@cygnus.com> To: garden-pa-list@wps.com, matthew@echo.com, tomj@fido.wps.com Subject: Re: outage last night... Status: OR We just had another brief outage. This was was more severe: the livingston router in 814 was not responsive EVEN ON THE LOCAL NET. Fortunately, someone was in 814, and I was able to reboot from the console. After a few seconds, the net was back to "normal." This happened too closely to an experiment I was trying for my comfort; I had done a "ping -s 2000 gw-liv.cygnus.com" at about the same time the router died (or, rather, just beforehand). Matthew Kaufman (associated with the Santa Cruz group), who has been doing some experimentation with my system and its networking, has also experienced some problems with both large packets, and medium-sized back-to-back packets. I think that we might want to either get in touch with livingston, and/or ask the livingston mailing list. Sean. From matthew@klinzhai.echo.com Sun Jun 27 15:43:43 1993 Received: from echo-slip.UCSC.EDU by fido.wps.com (5.67/9999.1) id AA00522; Sun, 27 Jun 93 15:43:38 -0700 Received: by klinzhai.echo.com (4.1/SMI-4.1.1) id AA17785; Sun, 27 Jun 93 15:44:29 PDT Date: Sun, 27 Jun 93 15:44:29 PDT From: matthew@klinzhai.echo.com (matthew kaufman) Message-Id: <9306272244.AA17785@klinzhai.echo.com> To: tomj@fido.wps.com Subject: TLG link problem Cc: gnu@toad.com, queue@klinzhai.echo.com, sef@kithrup.com Status: OR I have been doing some testing to try to figure out why telnet sessions from where i am to kithrup.com tend to hang. I have isolated the problem to the following: single large packets, or several smaller packets sent in rapid succession, are not reliably delivered over the link between gw-tis-ethernet.cygnus.com and gw-liv.cygnus.com. ONLY in the gw-liv TO gw-tis-ethernet direction. about 20%-30% are lost and never delivered. I've been able to prove that this is exactly what's happening by using 'ping' and then also reading the SNMP counters on gw-liv to see if the echo requests arrive vs. the replies finally getting back to me. this could be a router problem with either router, a csu/dsu problem or a pacbell problem. i would like to hear about how this problem turns out, because we have been planning on using a nearly identical configuration for the link to santa cruz, and if it turns out that there's a router problem, we'll need to find something more reliable. -matthew