I noticed there's already a dependency on velocity - that's great news
since I had hoped to replace a lot of ugly java code with a template. If
nothing else it would be easier to maintain. It's let me do some rapid
prototyping with the goal of both finding the best presentation (at least
for me) and to identify and remove unnecessary code. I have to step away
from other tasks for a few days but should have something good enough for a
POC pull request by early next week. This can also include sample output -
I've been focused on correctness over brevity and the current
implementation is creating images too large to attach here. (The brevity
would be handled by omitting default values and reducing whitespace - the
latter makes it easier to read by with 100+ dependencies it can make the
difference between whether something can be attached to email or not.
This will require the addition of at least one new property -
'templateName'. It can be optional (use a defautt template), or a
predefined one. It may be too risky to allow the user to specify an
arbitrary file, esp. since it's easy to clone the project and add your own
template(s) to the jar. The same thing applies to the velocity macro
libraries - they make it very easy to separate the structural and
presentation elements but allowing inclusion of arbitrary macros would be a
security risk.
Proposed code changes
1. There's a new package - org.apache.maven.dependency.utils.templates.
It's generic (templates, not velocity) so it's easy to add additional
template engines in the future.
2. There's a few new classes. I have an extended NodeProperties for a few
values that only make sense in this context, e.g., the 'port' used in the
dot 'edges'. I suspect it will be easiest to just add one or two new
properties to that class.
3. The second new class is similar to NodeProperties but is focused on the
groupId instead of the individual dependencies. For now this is in
'templates' but it might make sense to move it.
4. The third new class contains configurable style information. Fontnames
and sizes, colors to use, etc. Hmmm... this could also be handled via a
velocity macro...
5. Plus the helper class that actually creates and initializes the Velocity
engine and produces the final results.
There's also new resources - /templates/{format}/{templateName}. 'format'
would be 'dot', 'json', etc. We could add 'velocity' to the path but I
don't think that's necessary since there's no other templating engines on
the horizon and the fomats can be identified by file extension (.vm, .j2,
...)
Plus...
I'm always looking at the list of dependencies. I know this is a standalone
app, not a long-running service, so there's not the same risks when you
have a long list of dependencies. However I think it's still worth
considering since maven is so widely used in CI/CD pipelines.
I mentioned 'structural' vs 'presentation' aspects earlier. I don't know
how widely jackson is used but if it's only used to generate the JSON
output then it should be easy to replace that with a velocity template.
PARSING JSON has lots of dragons but GENERATING it is a lot easier, esp.
with the simple data structures used here.
The same thing may be true of GraphML and the other formats.
Plus...
The more I look at this the more I think there should be a clear
recognition that both list and tree formats should support both "by maven
dependency" and "by package name" layouts. For lists it would be a simple
sort by groupId. That makes it easy to check for consistent versions. For
trees the basic element would be each groupId with the artifacts in a
simple table. These blocks can still be ranked by the highest rank element.
I'm still experimenting with the dependency edges but it looks like a link
to the groupId's table, vs. individual artifacts, is a good balance between
seeing the maven dependencies vs. clutter. Ditto removing all dependencies
within a groupId. It may be possible to retain the outward bound link from
individual artifactId to the groupId.
This raised the possibility of a few new options.
The first is the ability to specify an arbitrary dependency (or groupid?)
as the root of the tree. For instance Jackson is built from multiple
modules and with this approach the exact dependencies will be lost. If we
can have a graph of just the Jackson interdependencies or the Jackson
dependencies + external dependencies it would make the trees even more
useful since they'll be small enough and focused enough to add to the
javadoc, package-info, or potentially even a standalone image in the source
directory. (I'm starting to inclulde README.md files since they're so
easily seen when browsing github in a browser.)
The second is the ability to collapse (groupId) dependencies. You might be
interested in the full tree but don't care about the details of some of the
large blocks, e.g., Jackson or org.apache.commons. For your purposes it's
enough to reduce the entire set of dependencies to a box about the same
size as the others. All of the inbound and outbound dependencies would
still be present but they would end at the collapsed dependency.
I need to do a little bit more research but I suspect this is a case that
shows why many of us have moved to other templates. In jinjava (and
thymeleaf?) we can define our own functions. This can dramatically reduce
the amount of cut-and-paste code in the template. Velocity seems to be
limited to macros. This isn't a big problem if you're doing something like
looking up the text color to use based on the maven scope but it makes it
harder to handle some dynamic structural elements. e.g., in dot format the
links can use either a "node" or a "node":port (note lack of quotes). The
former goes to the overall element while the latter goes to a specific
element in a list/table.
Bear
On Mon, Mar 24, 2025 at 9:38 PM Bear Giles <bgi...@coyotesong.com> wrote:
> One of the benefits (to me) is that it makes mismatches in dependency
> versions stand out.
>
> I've been able to figure out how to rotate the graph yet keep it
> relatively (cough) narrow. There's still a few unwanted empty boxes but it
> now has nested groupIds. If you look carefully at the com.faster.jackson
> you'll notice that there's one subgroup that has a single version but two
> other subgroups with multiple versions. Earlier I could see that these
> dependencies were pulled in from two locations and that's why there was a
> difference. Even if couldn't see the exact reason I know that this larger
> group of dependencies usually have the same version and if not I should
> take a closer look.
>
> At other places I've had to spend some time tracking down unnecessary
> transitive dependencies and fix them with a combination of "optional" and
> exclusions. Something like this would probably be easier to examine than a
> text listing of several hundred lines.
>
> (BTW the scope and 'optional' will be encoded as color and italics,
> respectively.)
>
> The latest version is attached. It's clear that it make sense to move
> "common-*" under the "org.apache.commons" group. I'm not sure about the
> org.google group - doing this kept some dependencies in the same subgraph
> but it also brought in some one-offs.
>
> [image: tree2.png]
>
>
> On Mon, Mar 24, 2025 at 7:37 PM Tomo Suzuki <suzt...@google.com.invalid>
> wrote:
>
>> That’s a beautiful diagram. Would you share the background you wanted to
>> visualize it in that way? Did it get some tasks done?
>>
>> Regards,
>> Tomo
>>
>> On Mon, Mar 24, 2025 at 19:02 Bear Giles <bgi...@coyotesong.com> wrote:
>>
>> > I should be clear - the glitch is the column of empty boxes above the
>> root
>> > dependency. They should not appear.
>> >
>> > The grayed areas, with one exception, have the same groupId. If they
>> also
>> > have the same version then the artifacts will be listed in a single
>> stack.
>> > The 'type' is omitted if it's "jar", otherwise it will be included on
>> that
>> > line.
>> >
>> > If there are multiple versions in the cluster then there will be
>> > multiple "records" with same structure as the standalone dependencies.
>> The
>> > only difference is that they''ll drop the groupId.
>> >
>> > The one exception is the "Database Drivers" in the bottom left - that's
>> an
>> > experiment specific to this pom - sometimes you might have a group
>> > independent dependencies that serve the same role, e.g., database JDBC
>> > drivers. Does it make sense to put them into their own cluster?
>> >
>> > Bear
>> >
>> > On Mon, Mar 24, 2025 at 4:52 PM Bear Giles <bgi...@coyotesong.com>
>> wrote:
>> >
>> >> The maven-dependency-plugin has a '.dot' option but because of the
>> layout
>> >> it's useless for anything other than the most trivial dependency
>> trees. I
>> >> know we can't replace existing functionality but this could be added as
>> >> "dot2", or a 'variant" property could be added to the mojo.
>> >>
>> >> There are some few obvious tweaks but some of them hit the limitations
>> of
>> >> the graphviz
>> >> library, or at least my understand of it. However I think I have
>> >> something very close to a good alternative to the standard layout - I
>> just
>> >> need to ask the graphviz people as question first.
>> >>
>> >> I've attached a sample pom and the generated png file. This is the pom
>> I
>> >> use for an application that creates a document listing the database
>> >> metadata for multiple databases.
>> >>
>> >> Bear Giles
>> >> [image: tree2.png]
>> >>
>> >
>>
>
