First of all thanks to the initial posts from below blogs that helped me to confirm and track the imminent storage issues that happened in our prod Cluster with 6.0U2 and HP-3Par.
Initially, the PE warnings/errors that are found with 6.0U2 with Hp-3Par have been neglected assuming these are just some bunch of warnings with the LUN Number 256 that is visible from HP-3Par even though we are not using Virtual Volume Feature. But, eventually these PE Lun’s have caused Production down when we tried to add new Datastores which initiated a rescan and two ESXi hosts went to Hung state as they can’t handle how to deal with the more than a few seconds of I/O Latency.
As mentioned by Virtual-Allan, this has been fixed in 3-Par by a FW patch, but interestingly HP never made this a recomended patch level. Also, this issue as mentioned is applicable to other storage arrays too that are presenting the PE LUN’s(Better to check for the anamolies during Rescan). Intrestingly we moved with changing the Max Disk Lun Setting to 250 in ESXi so that it can’t scan the PE Lun until our Storage team fixes the issue.