Rohit, I'll checkout out your work about using XML instead of @Component. However, I'm not sure if the memory leak issue is related with auto-scan at all. I'm now doing a heap analysis and will update with the result
Kelven On 2/21/13 7:02 AM, "Rohit Yadav" <bhais...@apache.org> wrote: >Okay Chip, posted a comment on the issue. > >Hi Kelven, you want to check my fix. Thanks. > >Regards. > >On Thu, Feb 21, 2013 at 8:17 PM, Chip Childers ><chip.child...@sungard.com> wrote: >> >> Can you coordinate with Kelvin on this? He's got CLOUDSTACK-1276, and >> had previously mentioned this exact same thing (using the xml file vs. >> annotations). >> >> On Thu, Feb 21, 2013 at 07:26:42PM +0530, Rohit Yadav wrote: >>> Just to add, we will have separate xmls for every kind of profile, I >>> can make every plugin have its own spring xml which is copied and >>> injected to the web.xml comments, suggestions, flames? >>> >>> Possible future experiments: >>> - Spring xmls artefacts based >>> - Have a way to fix AOP, don't add pointcuts on all classes, methods >>> - Make DI framework replaceable in CloudStack, I would go for Guice ;) >>> >>> Regards. >>> >>> On Thu, Feb 21, 2013 at 7:22 PM, Rohit Yadav <bhais...@apache.org> >>>wrote: >>> > I'm experimenting with removing all @Components and moving beans into >>> > an xml that could be read by spring to do DI. >>> > This decreases the load time and >>> > >>> > Errors: >>> > - Abstract classes annotated as spring components >>> > - Annotations for project which are not included in either build >>> > profiles: usage, simulator, oss (default) and nonoss >>> > - Some project don't even build or are not part of any build >>> > artefacts, looks like the @Component annotation was done in a hurry >>>:) >>> > - Interfaces being annotated as spring components >>> > - Prasanna, Abhi and I spoke with Spring gurus Vijay V and Vaibhav on >>> > how to fix this bloat, so in applicationContext.xml we do some aop >>> > stuff to have transaction stuff we get from @DB annotation and this >>> > needs fixing as it's applying the following for all classes and >>> > methods (dude bloat, run! :) >>> > >>> > <aop:config proxy-target-class="true"> >>> > <aop:aspect id="dbContextBuilder" >>>ref="transactionContextBuilder"> >>> > <aop:pointcut id="captureAnyMethod" >>> > expression="execution(* *(..))" >>> > /> >>> > >>> > Methodology and results: >>> > I wrote a python program that would go through the whole source code >>> > and fix all java files (except test, awsapi) who have @Component and >>> > create bean rules in an xml file. Next I had to manually fix few >>> > things (see error above) and I got the mgmt server running, the load >>> > time had improved a lot (little faster now) after disabling >>> > autoscanning and just using the xml generated. I see the benefit of >>> > just using the xml and not having spring as a build time dependency >>> > (we're using @Component in our classes so :). >>> > >>> > This was an experiment mostly because a lot of folks are having >>>spring >>> > memory issues, especially Prasanna's devcloud-ci cannot run and check >>> > our commits as memory kills dom0. So, if no one objects may I go >>>ahead >>> > and commit it? For me the jvm took about 650MB real memory on osx. >>> > >>> > Right now I made it for all build profiles, nonoss being the >>>superset, >>> > my experiment can be checked out from here: >>> > https://github.com/bhaisaab/cloudstack/tree/spring-annotation-to-xml >>> > >>> > Regards. >>> > >>> > The python program, in case anyone wants to replicate or is curious: >>> > (defaulter is just a file containing result of: file . | grep java$ > >>> > defaulter) >>> > f = open('defaulter', 'r') >>> > files = f.read().split('\n')[:-1] >>> > files = filter(lambda x: not x.startswith('./awsapi') and >>> > x.find('/test/') == -1, files) >>> > xmldata = "" >>> > bean = """ <bean id="%s" class="%s"/>\n""" >>> > xml = """<!-- >>> > Licensed to the Apache Software Foundation (ASF) under one >>> > or more contributor license agreements. See the NOTICE file >>> > distributed with this work for additional information >>> > regarding copyright ownership. The ASF licenses this file >>> > to you under the Apache License, Version 2.0 (the >>> > "License"); you may not use this file except in compliance >>> > with the License. You may obtain a copy of the License at >>> > >>> > http://www.apache.org/licenses/LICENSE-2.0 >>> > >>> > Unless required by applicable law or agreed to in writing, >>> > software distributed under the License is distributed on an >>> > "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY >>> > KIND, either express or implied. See the License for the >>> > specific language governing permissions and limitations >>> > under the License. >>> > --> >>> > <beans xmlns="http://www.springframework.org/schema/beans"; >>> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; >>> > xmlns:context="http://www.springframework.org/schema/context"; >>> > xmlns:tx="http://www.springframework.org/schema/tx"; >>> > xmlns:aop="http://www.springframework.org/schema/aop"; >>> > xsi:schemaLocation="http://www.springframework.org/schema/beans >>> > >>> > http://www.springframework.org/schema/beans/spring-beans-3.0.xsd >>> > http://www.springframework.org/schema/tx >>> > >>>http://www.springframework.org/schema/tx/spring-tx-3.0.xsd >>> > http://www.springframework.org/schema/aop >>> > >>> > http://www.springframework.org/schema/aop/spring-aop-3.0.xsd >>> > http://www.springframework.org/schema/context >>> > >>> > >>>http://www.springframework.org/schema/context/spring-context-3.0.xsd";> >>> > >>> > %s >>> > </beans> >>> > """ >>> > misses = [] >>> > for file in files: >>> > name = file.split('/')[-1].split('.')[0] >>> > name = name[0].lower() + name[1:] >>> > pkgname = "" >>> > idx = 0 >>> > paths = file.split('/')[1:] >>> > paths[-1] = paths[-1].split('.')[0] >>> > for str in paths: >>> > if str == 'org' or str == 'com': >>> > pkgname = '.'.join(paths[idx:]) >>> > idx += 1 >>> > f = open(file, 'r') >>> > data = f.read() >>> > f.close() >>> > idx = data.find('@Component') >>> > if idx == -1: >>> > misses.append(file) >>> > continue >>> > string = '@Component' >>> > if data[len(string)+idx] == '(': >>> > endidx = data.find(')', len(string)+idx) >>> > name = >>>data[len(string)+idx+1:endidx].split('=')[-1].replace('"', '') >>> > name = name.replace("'", '') >>> > data = data[:idx] + data[endidx+2:] >>> > else: >>> > data = data.replace(string, '') >>> > data = data.replace('import >>> > org.springframework.stereotype.Component;\n', '') >>> > f = open(file, 'w') >>> > f.write(data) >>> > f.close() >>> > print name, pkgname >>> > xmldata += bean % (name, pkgname) >>> > xmldata += "\n" >>> > >>> > xml = xml % xmldata >>> > f = open('./client/tomcatconf/defaultComponentContext.xml.in', 'w') >>> > f.write(xml) >>> > f.close() >>> > >>> > f = open('misses', 'w') >>> > f.write('\n'.join(misses)) >>> > f.close() >>>
