本文共 6912 字，大约阅读时间需要 23 分钟。

在WAS7.0.0.13版本上分布式发布ejb和web，web和ejb不在一个集群，且不在一个server哦情况下，web调用EJB的时候出现OOM错误，报错信息如下：

[11-3-1 3:32:06:132 CST] 00000021 SystemOut     O ADMINISTRATOR:123456:0:1:undefined:192.168.0.103:CrbGu3tv3gf37HzkT5r3BW7::[11-3-1 3:32:11:331 CST] 00000021 UserManagerDe E com.xx.xxxxxx.web.UserManagerDefaultImpl loginIn java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 	java.lang.OutOfMemoryError: 	>> SERVER (id=4773e3aa, host=zmt400) TRACE START:	>>    java.lang.OutOfMemoryError	>>	 at com.ibm.rmi.iiop.CDRReader.readBytesForString(CDRReader.java:2296)	>>	 at com.ibm.rmi.iiop.CDRReader.readStringOrIndirection(CDRReader.java:489)	>>	 at com.ibm.rmi.iiop.CDRReader.read_codebase_URL(CDRReader.java:2890)	>>	 at com.ibm.rmi.iiop.CDRReader.fast_read_value(CDRReader.java:1910)	>>	 at com.ibm.rmi.iiop.CDRReader.read_value(CDRReader.java:2017)	>>	 at com.xx.xxxx.common.service.ejb._EJSRemoteStatelesscom_xx_xxxxx_common_service_e_d15d50c6_Tie.loginIn__com_xx_xxxxx_privilege_UserInfoInterface__CORBA_WStringValue__CORBA_WStringValue__long__long_long__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue(_EJSRemoteStatelesscom_xx_xxxxx_common_service_e_d15d50c6_Tie.java:206)	>>	 at com.xxx.xxxxx.common.service.ejb._EJSRemoteStatelesscom_xx_xxxxx_common_service_e_d15d50c6_Tie._invoke(_EJSRemoteStatelesscom_xx_xxxxxx_common_service_e_d15d50c6_Tie.java:127)	>>	 at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:622)	>>	 at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:475)	>>	 at com.ibm.rmi.iiop.ORB.process(ORB.java:513)	>>	 at com.ibm.CORBA.iiop.ORB.process(ORB.java:1574)	>>	 at com.ibm.rmi.iiop.Connection.respondTo(Connection.java:2841)	>>	 at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2714)	>>	 at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:63)	>>	 at com.ibm.ejs.oa.pool.PooledThread.run(ThreadPool.java:118)	>>	 at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1563)	>> SERVER (id=4773e3aa, host=zmt400) TRACE END.[11-3-1 3:32:11:341 CST] 00000021 BaseServer    E com.xx.xxxxx.web.BaseServer processLogin 登录错误                                 java.lang.RuntimeException: java.rmi.ServerError: Error occurred in server thread; nested exception is: 	java.lang.OutOfMemoryError: 	>> SERVER (id=4773e3aa, host=zmt400) TRACE START:	>>    java.lang.OutOfMemoryError	>>	 at com.ibm.rmi.iiop.CDRReader.readBytesForString(CDRReader.java:2296)	>>	 at com.ibm.rmi.iiop.CDRReader.readStringOrIndirection(CDRReader.java:489)	>>	 at com.ibm.rmi.iiop.CDRReader.read_codebase_URL(CDRReader.java:2890)	>>	 at com.ibm.rmi.iiop.CDRReader.fast_read_value(CDRReader.java:1910)	>>	 at com.ibm.rmi.iiop.CDRReader.read_value(CDRReader.java:2017)	>>	 at com.xx.xxxxx.common.service.ejb._EJSRemoteStatelesscom_xx_xxxx_common_service_e_d15d50c6_Tie.loginIn__com_xx_xxxxxx_privilege_UserInfoInterface__CORBA_WStringValue__CORBA_WStringValue__long__long_long__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue(_EJSRemoteStatelesscom_xx_xxxxxx_common_service_e_d15d50c6_Tie.java:206)	>>	 at com.xx.xxxx.common.service.ejb._EJSRemoteStatelesscom_xx_xxxxxx_common_service_e_d15d50c6_Tie._invoke(_EJSRemoteStatelesscom_xx_xxxxx_common_service_e_d15d50c6_Tie.java:127)	>>	 at com.ibm.CORBA.iiop.ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:622)	>>	 at com.ibm.CORBA.iiop.ServerDelegate.dispatch(ServerDelegate.java:475)	>>	 at com.ibm.rmi.iiop.ORB.process(ORB.java:513)	>>	 at com.ibm.CORBA.iiop.ORB.process(ORB.java:1574)	>>	 at com.ibm.rmi.iiop.Connection.respondTo(Connection.java:2841)	>>	 at com.ibm.rmi.iiop.Connection.doWork(Connection.java:2714)	>>	 at com.ibm.rmi.iiop.WorkUnitImpl.doWork(WorkUnitImpl.java:63)	>>	 at com.ibm.ejs.oa.pool.PooledThread.run(ThreadPool.java:118)	>>	 at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1563)	>> SERVER (id=4773e3aa, host=zmt400) TRACE END.

当然第一反应是应用写的有问题，ejb里面有内存泄露或者threadpool开太大，后来检查未发现异常。

收集CORE文件和HEAPDUMP文件分析，上PP。

core文件分析如下：

1TISIGINFO     Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" received ..................1XMCURTHDINFO  Current Thread DetailsNULL           ----------------------3XMTHREADINFO      "ORB.thread.pool : 0" TID:0x0000000001ECF600, j9thread_t:0x0000000011EA3360, state:R, prio=53XMTHREADINFO1            (native thread ID:0x5C7C, native priority:0x5, native policy:UNKNOWN)4XESTACKTRACE          at com/ibm/rmi/iiop/CDRReader.readBytesForString(CDRReader.java:2279)4XESTACKTRACE          at com/ibm/rmi/iiop/CDRReader.readStringOrIndirection(CDRReader.java:472)4XESTACKTRACE          at com/ibm/rmi/iiop/CDRReader.read_codebase_URL(CDRReader.java:2852)4XESTACKTRACE          at com/ibm/rmi/iiop/CDRReader.fast_read_value(CDRReader.java:1893)4XESTACKTRACE          at com/ibm/rmi/iiop/CDRReader.read_value(CDRReader.java:2000)4XESTACKTRACE          at com/xx/xxxxxx/common/service/ejb/_EJSRemoteStatelesscom_xx_xxxxxx_common_service_e_d15d50c6_Tie.loginIn__com_xx_xxxx_privilege_UserInfoInterface__CORBA_WStringValue__CORBA_WStringValue__long__long_long__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue__CORBA_WStringValue(_EJSRemoteStatelesscom_xx_xxxxx_common_service_e_d15d50c6_Tie.java:206)4XESTACKTRACE          at com/xx/xxxxxx/common/service/ejb/_EJSRemoteStatelesscom_xx_xxxxx_common_service_e_d15d50c6_Tie._invoke(_EJSRemoteStatelesscom_xx_xxxxxx_common_service_e_d15d50c6_Tie.java:127)4XESTACKTRACE          at com/ibm/CORBA/iiop/ServerDelegate.dispatchInvokeHandler(ServerDelegate.java:622)4XESTACKTRACE          at com/ibm/CORBA/iiop/ServerDelegate.dispatch(ServerDelegate.java:475)4XESTACKTRACE          at com/ibm/rmi/iiop/ORB.process(ORB.java:504)4XESTACKTRACE          at com/ibm/CORBA/iiop/ORB.process(ORB.java:1571)4XESTACKTRACE          at com/ibm/rmi/iiop/Connection.respondTo(Connection.java:2771)4XESTACKTRACE          at com/ibm/rmi/iiop/Connection.doWork(Connection.java:2640)4XESTACKTRACE          at com/ibm/rmi/iiop/WorkUnitImpl.doWork(WorkUnitImpl.java:63)4XESTACKTRACE          at com/ibm/ejs/oa/pool/PooledThread.run(ThreadPool.java:118)4XESTACKTRACE          at com/ibm/ws/util/ThreadPool$Worker.run(ThreadPool.java:1527)

 heapdump分析如下：

发现CORE文件里面很奇怪，Free Java Heap 有2,104,902,904，98%空闲，但是在内存段分析中，Object即heap占用2,147,483,648 100%占用，heapdump里面分析占用heap内存42,565,544，和core文件的第一种说法匹配，即98% free，排除应用占用内存的可能。

分析了下出错的代码行 com.ibm.rmi.iiop.CDRReader.readBytesForString，发现是由web端调用ejb的时候，从ejb容器传递ejb stud的时候出现问题，好，那就避开这个，我将应用发布到同一个server上，这样ejb应该优化为本地调用，验证成功。

那为了解决分布式调用的问题，我还要想一个办法，让系统在分布式情况下也不用传递ejb stud，这让我想到了早期中间件需要手动打stud然后引入web容器的事情，查了下infocenter，找到{WAS_HOME}\bin\ejbdeploy.sh命令。

ejbdeploy tytim.ear . tytim_stub.ear

会在当前目录产生tytim_stub.ear，将这个ear中的ejb相关jar覆盖到.war里面的lib中，问题解决。

已联系IBM技术人员，看看是什么原因导致ejb stud传递的时候byte数组长度出现异常，等有了回复，在补充上来。

转载请注明原始地址：