GroupStudy.com GroupStudy.com - A virtual community of network engineers
 Home  BookStore  StudyNotes  Links  Archives  StudyRooms  HelpWanted  Discounts  Login
Re: Partial Network Outage on Catalyst WS-C6509-E using dual Sup posted 09/28/2008
[Chronological Index] [Thread Index] [Top] [Date Prev][Date Next] [Thread Prev][Thread Next]


Thanks Pierre.

I might have forgotten to add the "test ?" options that tend to vary per
platform for some additional insight.

The show proc | mem extended options might also shed some light if there is
a hung process or a memory leak too.




On Sat, Sep 27, 2008 at 7:19 PM, Pierre-Alex GUANEL <paguanel@xxxxxxxxx>wrote:

> Hi mrt,
>
> Just to add to Darby's list, you can do a "show diagnostic result module 6"
> to see the nature of the failed diagnostic test ....
>
> HTH
>
> PA
> ----- Original Message ----- From: "Darby Weaver" <ccie.weaver@xxxxxxxxx>
> To: "john matijevic" <john.matijevic@xxxxxxxxx>
> Cc: "Mister T" <romantic24hrs@xxxxxxxxx>; "ccielab" <
> ccielab@xxxxxxxxxxxxxx>
> Sent: Saturday, September 27, 2008 11:31 PM
> Subject: Re: Partial Network Outage on Catalyst WS-C6509-E using dual Sup
>
>
>
> A few things that you might consider:
>>
>> 1. Where is this line:
>>
>> %FABRIC-SP-5-FABRIC_MODULE_ACTIVE: The Switch Fabric Module in slot 5
>> became active.
>>
>> 2. What IOS Version do you have on yuor supervisor?  Have you checked it
>> for
>> any issues or bugs?  Was it upgraded recently?  Why?
>>
>> 3. Have you consulted TAC?
>>
>> 4. Have you considered configured the supervisor to offload the crashdump
>> to
>> an FTP server so you'll have something tangible for TAC?
>>
>> 5. Have you prepared a show tech-support for TAC to analyze?
>>
>> 6. Are there any other events surrounding this issue and exactly how many
>> time and how often is it happening?
>>
>> 7. Have you checked the flash for any crash_dump information?
>>
>> Let me know if I can be of further assistance.
>>
>> Darby Weaver
>>
>> Just some ideas off the top.
>>
>> On Sat, Sep 27, 2008 at 5:30 PM, john matijevic <john.matijevic@xxxxxxxxx
>> >wrote:
>>
>> Hello,
>>> Try replacing the Sup modules.
>>> Sincerely,
>>> John
>>>
>>> On Sat, Sep 27, 2008 at 3:38 AM, Mister T <romantic24hrs@xxxxxxxxx>
>>> wrote:
>>>
>>> > Dear expert,
>>> >
>>> > I have problem experiencing partial network outages in my one Catalyst
>>> > 6509-E. It happens many times in almost 2 months, and when it happens
>>> > partial network outage takes about 10 minutes (LAN and WAN traffic >
>>> down).
>>> >
>>> > I have a suspect that something is not right for the dual supervisor
>>> > redundancy (when doing switchover). I also noticed that the standby Sup
>>> in
>>> > slot 6 shows 'Minor Error'. I went to the 'Error Message Decoder' at
>>> > cisco.com to interpret the error messages from the log, but it says:
>>> no
>>> > action is required. For details i enclose the log from the switch.
>>> >
>>> > ------------------ show module ------------------
>>> >
>>> >
>>> > Mod Ports Card Type                              Model
>>>  Serial
>>> > No.
>>> > --- ----- -------------------------------------- ------------------
>>> > -----------
>>> >  2   16  SFM-capable 16 port 1000mb GBIC        WS-X6516-GBIC      -
>>> >  3   16  SFM-capable 16 port 1000mb GBIC        WS-X6516-GBIC      -
>>> >  4   16  SFM-capable 16 port 1000mb GBIC        WS-X6516A-GBIC     -
>>> >  5    2  Supervisor Engine 720 (Active)         WS-SUP720-3B       -
>>> >  6    2  Supervisor Engine 720 (Hot)            WS-SUP720-3B       -
>>> >  7   16  SFM-capable 16 port 10/100/1000mb RJ45 WS-X6516-GE-TX     -
>>> >  8   48  SFM-capable 48 port 10/100/1000mb RJ45 WS-X6548-GE-TX     -
>>> >
>>> > Mod MAC addresses                       Hw    Fw           Sw
>>> > Status
>>> > --- ---------------------------------- ------ ------------ ------------
>>> > -------
>>> >  2  0001.6463.0df8 to 0001.6463.0e07   4.4   7.2(1)       8.5(0.46)RFW
>>> > Ok
>>> >  3  0009.11e3.05a4 to 0009.11e3.05b3   5.0   6.3(1)       8.5(0.46)RFW
>>> > Ok
>>> >  4  0019.aa98.6a20 to 0019.aa98.6a2f   4.5   7.2(1)       8.5(0.46)RFW
>>> > Ok
>>> >  5  0014.a982.4ba0 to 0014.a982.4ba3   5.2   8.4(2)       12.2(18)SXF9
>>> > Ok
>>> >  6  0017.9441.c344 to 0017.9441.c347   5.2   8.4(2)       12.2(18)SXF9
>>> > Ok
>>> >  7  001a.6d87.0cf0 to 001a.6d87.0cff   2.8   6.3(1)       8.5(0.46)RFW
>>> > Ok
>>> >  8  0009.7c46.ee4a to 0009.7c46.ee79  10.1   7.2(1)       8.5(0.46)RFW
>>> > Ok
>>> >
>>> > Mod  Sub-Module                  Model              Serial       Hw
>>> > Status
>>> > ---- --------------------------- ------------------ ----------- -------
>>> > -------
>>> >  5  Policy Feature Card 3       WS-F6K-PFC3B       -             2.3
>>>  Ok
>>> >  5  MSFC3 Daughterboard         WS-SUP720          -            2.5 >
>>> Ok
>>> >  6  Policy Feature Card 3       WS-F6K-PFC3B       -            2.3 >
>>> Ok
>>> >  6  MSFC3 Daughterboard         WS-SUP720          -            2.5 >
>>> Ok
>>> >
>>> > Mod  Online Diag Status
>>> > ---- -------------------
>>> >  2  Pass
>>> >  3  Pass
>>> >  4  Pass
>>> >  5  Pass
>>> >  6  Minor Error
>>> >  7  Pass
>>> >  8  Pass
>>> >
>>> >
>>> > ***************************************************
>>> > ****** Information of Last System Crash - SP ******
>>> > ***************************************************
>>> >
>>> >
>>> > Using sup-bootflash:crashinfo_20080915-101604.
>>> >
>>> > Writing crashinfo to bootflash:crashinfo_20080915-101604ot 6, >
>>> interfaces
>>> > are
>>> > now online
>>> >
>>> > Sep 15 17:02:34.005 WIB: %PFREDUN-SP-6-ACTIVE: Standby initializing for
>>> SSO
>>> > mode
>>> > Sep 15 17:02:34.005 WIB: %SYS-SP-3-LOGGER_FLUSHING: System pausing to
>>> > ensure
>>> > console debugging output.
>>> >
>>> > Sep 15 17:02:34.005 WIB: %PFREDUN-SP-6-ACTIVE: Standby initializing for
>>> SSO
>>> > mode
>>> > Sep 15 17:02:34.201 WIB: %SYS-SP-3-LOGGER_FLUSHED: System was paused >
>>> for
>>> > 00:00:00 to ensure console debugging output.
>>> >
>>> > Sep 15 17:02:35.525 WIB: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
>>> > configuration to the standby Router.
>>> > Sep 15 17:03:15.510 WIB: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option >
>>> is
>>> > off
>>> > for the fabric in slot 6.
>>> > Sep 15 17:03:15.606 WIB: %FABRIC-SP-5-FABRIC_MODULE_BACKUP: The Switch
>>> > Fabric Module in slot 6 became standby
>>> > Sep 15 17:03:16.434 WIB: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running
>>> Minimal
>>> > Diagnostics...
>>> > Sep 15 17:03:17.874 WIB: %DIAG-SP-6-DIAG_OK: Module 6: Passed Online
>>> > Diagnostics
>>> > Sep 15 17:03:18.114 WIB: %OIR-SP-6-INSCARD: Card inserted in slot 6,
>>> > interfaces are now online
>>> > Sep 15 17:07:55.371 WIB: %PFREDUN-SP-6-ACTIVE: Standby processor >
>>> removed
>>> or
>>> > reloaded, changing to Simplex mode
>>> > Sep 15 17:10:07.419 WIB: %PFREDUN-SP-6-ACTIVE: Standby initializing for
>>> SSO
>>> > mode
>>> > Sep 15 17:10:07.419 WIB: %SYS-SP-3-LOGGER_FLUSHING: System pausing to
>>> > ensure
>>> > console debugging output.
>>> >
>>> > Sep 15 17:10:07.419 WIB: %PFREDUN-SP-6-ACTIVE: Standby initializing for
>>> SSO
>>> > mode
>>> > Sep 15 17:10:07.611 WIB: %SYS-SP-3-LOGGER_FLUSHED: System was paused >
>>> for
>>> > 00:00:00 to ensure console debugging output.
>>> >
>>> > Sep 15 17:10:09.644 WIB: %PFINIT-SP-5-CONFIG_SYNC: Sync'ing the startup
>>> > configuration to the standby Router.
>>> > Sep 15 17:10:50.140 WIB: %FABRIC-SP-5-CLEAR_BLOCK: Clear block option >
>>> is
>>> > off
>>> > for the fabric in slot 6.
>>> > Sep 15 17:10:50.232 WIB: %FABRIC-SP-5-FABRIC_MODULE_BACKUP: The Switch
>>> > Fabric Module in slot 6 became standby
>>> > Sep 15 17:10:51.560 WIB: %DIAG-SP-6-RUN_MINIMUM: Module 6: Running
>>> Minimal
>>> > Diagnostics...
>>> > Sep 15 17:10:53.020 WIB: %DIAG-SP-6-DIAG_OK: Module 6: Passed Online
>>> > Diagnostics
>>> > Sep 15 17:10:53.216 WIB: %OIR-SP-6-INSCARD: Card inserted in slot 6,
>>> > interfaces are now online
>>> > Sep 15 17:15:30.617 WIB: %PFREDUN-SP-6-ACTIVE: Standby processor >
>>> removed
>>> or
>>> > reloaded, changing to Simplex mode
>>> >
>>> >
>>> > Any help to resolve this problem would be appreciated.
>>> >
>>> > Regards
>>> >
>>> > mrt
>>> >
>>> >
>>> > Blogs and organic groups at http://www.ccie.net
>>> >
>>> > _______________________________________________________________________
>>> > Subscription information may be found at:
>>> > http://www.groupstudy.com/list/CCIELab.html
>>>
>>>
>>> Blogs and organic groups at http://www.ccie.net
>>>
>>> _______________________________________________________________________
>>> Subscription information may be found at:
>>> http://www.groupstudy.com/list/CCIELab.html
>>>
>>
>>
>> Blogs and organic groups at http://www.ccie.net
>>
>> _______________________________________________________________________
>> Subscription information may be found at:
>> http://www.groupstudy.com/list/CCIELab.html