Excerpts from Maciej Bliziński's message of Sun Aug 21 03:27:08 -0400 2011:
> > As far as the catalog diff engine is concerned, both of these are > > the same. They'll appear as the drop of one package and the > > addition of a new one. The special case of _stub packages is > > handled in the pseudo code. The general case would require doing > > a reverse index of package names. The _stub handling rename > > detection could be extended to: > > > > * Build a package to catalog map, based on previous catalog, of > > every package we're going to alter. > > What's the definition of a package that can be altered? This is just an object that we thing needs database changes to reflect current state. The results of the catalog diff will determine this. > > * Do the same for the new catalog. > > * Iterate through new packages and see if the package name exists in > > the old catalog. > > * If we 'hit' in the index, add [p, q] to the rename list and remove > > them from the new and remove lists. > > I think that at this point, we know the area in which we can look > for corner cases. It'll be best to talk in code: if you have a unit > test with test cases, we can define the test cases and see what does > the diff engine output. Yes, that's the way to approach it. > > We'd still want to filter stubs to a separate list first as those > > packages shouldn't be displayed in the web db or mantis. > > I'm thinking that mantis should not contain the packages (whatever > the word 'package' means), but something that Dagobert calls > bundles. They are roughly equivalent to upstream projects. This is ideally what should happen. This could be grouped based on the SVN url that we embed. It's a massive change from what we have now though. If we want to go this route, we should simply mark the current stuff as legacy and start fresh with a) a normalized and sane db for the web and b) a bug tracker that is better able to meet our current (and hopefully future) needs. > >> 3. Only the pkgname changes (CSWfoodev/foo_dev → CSWfoo-dev/foo_dev) > > > > This should be handled by an update operation. As things are keyed on > > the catalog name, a change (based on version comparison) will trigger > > a refresh of file list, maintainer name, etc. Mantis cares not about > > package name so that won't be a problem. > > > > Reasonable? > > I expected that you'll say that this scenario is invalid / > forbidden. If the pkgname changes, you need to provide a _stub for > the catalogname. We need some kind of detection of invalid > scenarios, too. Yes, the high level code that I've written can detect several things. I'm starting to think that instead of an asynchronous job running independently that we should instead place this in serial right after generating a new catalog, but before mirror push. > Let's continue by reviewing test cases, how does that sound? Sounds good. I need to refactor my code such that it's a testable module. The catalog diff part is already, but the actionable items is not presently. I'll work on that tonight. The code is now capable of spitting out a list of high level action items though. I still need to get a test environment running, but the logic and unit testing of the high level part can proceed without that. Given the catalog in current (5.9/sparc) and the catalog in future/unstable (5.9/sparc), the following is the list that my code generates: Rename py_libxslt/CSWpylibxslt -> py_libxslt/CSWpy-libxslt Rename berkeleydb3_doc/CSWbdb3doc -> berkeleydb3_doc/CSWbdb3-doc Rename libserf_devel/CSWlibserf-devel -> libserf_dev/CSWlibserf-dev Rename pcre_devel/CSWpcre-devel -> libpcre_dev/CSWlibpcre-dev Rename zlib/CSWzlib -> libz1/CSWlibz1 Rename pm_x11protocol/CSWpmx11protocol -> pm_x11_protocol/CSWpm-x11-protocol Rename libxslt_devel/CSWlibxsltdevel -> libxslt_dev/CSWlibxslt-dev Rename libdbusdev/CSWlibdbusdev -> libdbus_dev/CSWlibdbus-dev Rename apr/CSWapr -> libapr1_0/CSWlibapr1-0 Rename libgnugetopt/CSWlibgnugetopt -> libgnugetopt0/CSWlibgnugetopt0 Rename id3lib/CSWid3lib -> libid3_utils/CSWlibid3-utils Rename libdnet/CSWlibdnet -> libdnet1/CSWlibdnet1 Rename screen/CSWscrn -> screen/CSWscreen Rename lzma/CSWlzma -> xz/CSWxz Rename eventlog/CSWeventlog -> eventlog_dev/CSWeventlog-dev Rename pm_version/CSWpmversion -> pm_version/CSWpm-version Rename cadaver/CSWcadav -> cadaver/CSWcadaver Rename apr_util/CSWapr-util -> libaprutil1_ldap/CSWlibaprutil1-ldap Rename pcre_rt/CSWpcrert -> libpcreposix0/CSWlibpcreposix0 Rename libiconv/CSWiconv -> libiconv2/CSWlibiconv2 Rename libevent_devel/CSWlibevent-devel -> libevent_dev/CSWlibevent-dev Rename sqlite3rt_stub/CSWsqlite3rt -> libsqlite3_0/CSWlibsqlite3-0 Rename libfbopenssl/CSWlibfbopenssl -> libfbopenssl0/CSWlibfbopenssl0 Rename glew/CSWglew -> libglew_dev/CSWlibglew-dev Rename python_devel/CSWpython-devel -> python_dev/CSWpython-dev Rename libdbus/CSWlibdbus -> libdbus1_3/CSWlibdbus1-3 Rename libffi/CSWlibffi -> libffi5/CSWlibffi5 Rename sqlite3devel_stub/CSWsqlite3devel -> libsqlite3_dev/CSWlibsqlite3-dev Rename libdatrie/CSWlibdatrie -> libdatrie1/CSWlibdatrie1 Register py_mysql/CSWpy-mysql Register idle_27/CSWidle27 Register libexpat0/CSWlibexpat0 Register libexpat1/CSWlibexpat1 Register libz_dev/CSWlibz-dev Register pm_test_object/CSWpm-test-object Register libaprutil1_dbd_freetds/CSWlibaprutil1-dbd-freetds Register pm_ppix_regexp/CSWpm-ppix-regexp Register python_27_tk/CSWpython27-tk Register python_27_dev/CSWpython27-dev Register berkeleydb3_dev/CSWbdb3-dev Register python_31_devel/CSWpython31-devel Register libdatrie_dev/CSWlibdatrie-dev Register opencsw_policy/CSWopencsw-policy Register libslp1/CSWlibslp1 Register libbz2_dev/CSWlibbz2-dev Register libpython3_1_1_0/CSWlibpython3-1-1-0 Register rb18_activerecord_3_0_7/CSWrb18-activerecord-3-0-7 Register libbz2_1_0/CSWlibbz2-1-0 Register py_xapian/CSWpy-xapian Register libglew1_6/CSWlibglew1-6 Register rb18_mysql_2_8_1/CSWrb18-mysql-2-8-1 Register rtmpdump/CSWrtmpdump Register libexpat_dev/CSWlibexpat-dev Register rb18_i18n_0_5_0/CSWrb18-i18n-0-5-0 Register pm_string_format/CSWpm-string-format Register libmp4v2_dev/CSWlibmp4v2-dev Register libpython2_7_1_0/CSWlibpython2-7-1-0 Register libiconv_dev/CSWlibiconv-dev Register rb18_activemodel_3_0_7/CSWrb18-activemodel-3-0-7 Register libwsutil0/CSWlibwsutil0 Register py_virtualenv/CSWpy-virtualenv Register libxslt1/CSWlibxslt1 Register xmms_faad2/CSWxmms-faad2 Register libapr_dev/CSWlibapr-dev Register py_docutils/CSWpy-docutils Register py_jinja2/CSWpy-jinja2 Register pm_ppix_utilities/CSWpm-ppix-utilities Register pm_ppi/CSWpm-ppi Register libaprutil_dev/CSWlibaprutil-dev Register libfaad2/CSWlibfaad2 Register libmp4v2_0/CSWlibmp4v2-0 Register libmp4v2_utils/CSWlibmp4v2-utils Register openslp_devel/CSWopenslp-devel Register libwireshark0/CSWlibwireshark0 Register rb_xapian/CSWrb-xapian Register proj_data/CSWproj-data Register libmpc2/CSWlibmpc2 Register libmpfr_dev/CSWlibmpfr-dev Register libfbopenssl_dev/CSWlibfbopenssl-dev Register libidn/CSWlibidn Register python_27/CSWpython27 Register libmysqlclient_r15/CSWlibmysqlclient-r15 Register libcharset1/CSWlibcharset1 Register libffi_dev/CSWlibffi-dev Register libclam6_dev/CSWlibclam6-dev Register libexslt0/CSWlibexslt0 Register pinentry_gtk2/CSWpinentry-gtk2 Register pcp/CSWpcp Register rb18_ratom_0_6_7/CSWrb18-ratom-0-6-7 Register libaprutil1_dbd_sqlite3/CSWlibaprutil1-dbd-sqlite3 Register libdnet_dev/CSWlibdnet-dev Register py_dbutils/CSWpy-dbutils Register pm_netaddr_ip/CSWpm-netaddr-ip Register libevtlog0/CSWlibevtlog0 Register idle_31/CSWidle31 Register pm_test_subcalls/CSWpm-test-subcalls Register libserf1_0/CSWlibserf1-0 Register xapian/CSWxapian Register rb18_activesupport_3_0_7/CSWrb18-activesupport-3-0-7 Register py_openssl/CSWpy-openssl Register libaprutil1_dbd_odbc/CSWlibaprutil1-dbd-odbc Register libproj0/CSWlibproj0 Register libtre5/CSWlibtre5 Register libgmpxx4/CSWlibgmpxx4 Register libmpfr4/CSWlibmpfr4 Register libgmp_dev/CSWlibgmp-dev Register py_pygments/CSWpy-pygments Register libxapian22/CSWlibxapian22 Register libiconv_utils/CSWlibiconv-utils Register libgmp10/CSWlibgmp10 Register rb18_builder_3_0_0/CSWrb18-builder-3-0-0 Register libmpc_dev/CSWlibmpc-dev Register py_subvertpy/CSWpy-subvertpy Register libid3_3_8_3/CSWlibid3-3-8-3 Register libwiretap0/CSWlibwiretap0 Register libfaad_dev/CSWlibfaad-dev Register libmysqlclient15/CSWlibmysqlclient15 Register libxapian_dev/CSWlibxapian-dev Register pm_email_address/CSWpm-email-address Register py_bridge/CSWpy-bridge Register python_31_tk/CSWpython31-tk Register tcllib/CSWtcllib Register rb18_json_1_5_3/CSWrb18-json-1-5-3 Register librtmp_dev/CSWlibrtmp-dev Register librtmp0/CSWlibrtmp0 Register pm_perl_ostype/CSWpm-perl-ostype Register proj_dev/CSWproj-dev Register libaprutil1_0/CSWlibaprutil1-0 Register python_31/CSWpython31 Register libdatrie_utils/CSWlibdatrie-utils Register pm_perl_critic/CSWpm-perl-critic Register cas_etcshells/CSWcas-etcshells Register py_sphinx/CSWpy-sphinx Register libgnugetopt_dev/CSWlibgnugetopt-dev Register tre_dev/CSWtre-dev Register libaprutil1_dbm_db/CSWlibaprutil1-dbm-db Register pm_pod_spell/CSWpm-pod-spell Register pm_dbi2/CSWpmdbi Register guilt/CSWguilt Register libid3_dev/CSWlibid3-dev Register rb18_libxml_1_1_4/CSWrb18-libxml-1-1-4 Update easytag/CSWeasytag Update enscript/CSWenscript Update asciidoc/CSWasciidoc Update lftp/CSWlftp Update xorriso/CSWxorriso Update expat/CSWexpat Update libmagic_data/CSWlibmagic-data Update libclam6/CSWlibclam6 Update git_completion/CSWgit-completion Update xjobs/CSWxjobs Update unrar/CSWunrar Update mysql5/CSWmysql5 Update memconf/CSWmemconf Update py_nose/CSWpy-nose Update gawk/CSWgawk Update dhcp/CSWdhcp Update cswclassutils/CSWcswclassutils Update imapproxy/CSWimapproxy Update proj/CSWproj Update ruby_augeas/CSWrubyaugeas Update mercurial/CSWmercurial Update py_tz/CSWpy-tz Update py_libmagic/CSWpy-libmagic Update geoip/CSWgeoip Update openssh_client/CSWosshclient Update gxmessage/CSWgxmessage Update lsof/CSWlsof Update gfile/CSWgfile Update sqlite3/CSWsqlite3 Update git_cvs/CSWgit-cvs Update python_test/CSWpython-test Update libgd_dev/CSWlibgd-dev Update py_xcbproto/CSWpyxcbproto Update clusterssh/CSWclusterssh Update pinentry/CSWpinentry Update libflac8/CSWlibflac8 Update clamav/CSWclamav Update bind_utils/CSWbindutils Update libgd2/CSWlibgd2 Update python/CSWpython Update wgetpaste/CSWwgetpaste Update mysql5devel/CSWmysql5devel Update idle/CSWidle Update mailscanner/CSWmailscanner Update dcc/CSWdcc Update elinks/CSWelinks Update bash/CSWbash Update zsh/CSWzsh Update bar/CSWbar Update gd/CSWgd Update vsftpd/CSWvsftpd Update git_svn/CSWgit-svn Update netcat/CSWnetcat Update py_hachoir_parser/CSWpy-hachoir-parser Update rlwrap/CSWrlwrap Update bind_chroot/CSWbindchroot Update xmlstarlet/CSWxmlstarlet Update gcal/CSWgcal Update libpcrecpp0/CSWlibpcrecpp0 Update pca/CSWpca Update libmagic_dev/CSWlibmagic-dev Update liblzma5/CSWliblzma5 Update bind_dev/CSWbind-dev Update mysql5client/CSWmysql5client Update mysql5rt/CSWmysql5rt Update liblzma_dev/CSWliblzma-dev Update git_doc/CSWgit-doc Update tnef/CSWtnef Update libflac_dev/CSWlibflac-dev Update openssl_rt/CSWosslrt Update ttcp/CSWttcp Update geoip_dev/CSWgeoip-dev Update lighttpd/CSWlighttpd Update flac/CSWflac Update py_graph/CSWpy-graph Update cas_initsmf/CSWcas-initsmf Update git_dev/CSWgit-dev Update python_tk/CSWpython-tk Update libpython2_6_1_0/CSWlibpython2-6-1-0 Update pymysql/CSWpymysql Update berkeleydb3/CSWbdb3 Update pm_netaddrip/CSWpmnetaddrip Update xmms_flac/CSWxmms-flac Update libmagic1/CSWlibmagic1 Update clamavsigs/CSWclamavsigs Update openssl_devel/CSWossldevel Update namebench/CSWnamebench Update tcsh/CSWtcsh Update bzip2/CSWbzip2 Update facter/CSWfacter Update pbzip2/CSWpbzip2 Update libbind/CSWlibbind Update miltergreylist/CSWmiltergreylist Update gitk/CSWgitk Update libgmp/CSWlibgmp Update git_emacs/CSWgit-emacs Update libevent2_0_5/CSWlibevent2-0-5 Update nicstat/CSWnicstat Update dhcp_dev/CSWdhcp-dev Update bind/CSWbind Update tmux/CSWtmux Update py_sqlobject/CSWpy-sqlobject Update faad2/CSWfaad2 Update git/CSWgit Update libxslt/CSWlibxslt Update dbus/CSWdbus Update openssl_utils/CSWosslutils Update libmpfr/CSWlibmpfr Update openssh/CSWossh Update openssl/CSWossl Update cmake/CSWcmake Update jpegoptim/CSWjpegoptim Update spamassassin/CSWspamassassin Update mgar/CSWmgar Update nginx/CSWnginx Update git_gui/CSWgit-gui Update xcbproto/CSWxcbproto Update pydocutils/CSWpydocutils Update pm_html_parser/CSWpm-html-parser Update cas_migrateconf/CSWcas-migrateconf Update phpmyadmin/CSWphpmyadmin Update pcre/CSWpcre Update pm_htmlparser/CSWpmhtmlparser Update libpcre0/CSWlibpcre0 Update unzoo/CSWunzoo Removing drupal/CSWdrupal Removing libclam6_devel/CSWlibclam6-devel Removing libsmi_mibs_stub/CSWlibsmimibs Removing mediawiki/CSWmediawiki Removing pm_dbi_stub/CSWpmdbi Removing buildbot/CSWbuildbot Removing zope/CSWzope Removing libidn_stub/CSWlibidn Removing py_atomixlib/CSWpy-atomixlib Removing mysql5bench/CSWmysql5bench Removing squirrelmail/CSWsquirrelmail Thanks -Ben -- Ben Walton Systems Programmer - CHASS University of Toronto C:416.407.5610 | W:416.978.4302 _______________________________________________ devel mailing list devel@lists.opencsw.org https://lists.opencsw.org/mailman/listinfo/devel
