It’s been about a month since my last website status update. In the time I’ve found to work on this, most of my focus has been on providing a smooth developer experience to better facilitate contributions, with some minor housekeeping items on the side. Here’s what has happened since then:
- *We have the first docs page on the new website:* https://ozone-site-v2.staged.apache.org/docs/developer-guide/project/release-guide Thanks Attila for helping to port this over from Confluence. Check this out to get an idea of what pages on the new website will look like when rendered and some of the features docusaurus provides to make pages easier to read. - *A CI pipeline for the website is in progress.* I have a POC on my fork that is being ported over to the main site in smaller PRs. This includes spell checking, markdown format checks, license header checks, building and running the site, and more. See HDDS-9601 <https://issues.apache.org/jira/browse/HDDS-9601> for an update on the progress. Once HDDS-10254 <https://issues.apache.org/jira/browse/HDDS-10254> HDDS-9866 <https://issues.apache.org/jira/browse/HDDS-9866> HDDS-9567 <https://issues.apache.org/jira/browse/HDDS-9567> and HDDS-10351 <https://issues.apache.org/jira/browse/HDDS-10351> are completed, we should have solid checks to maintain website format across the anticipated hundreds of pages. Thanks Attila for reviewing these changes! *HDDS-9569 <https://issues.apache.org/jira/browse/HDDS-9569> HDDS-9570 <https://issues.apache.org/jira/browse/HDDS-9570> HDDS-10354 <https://issues.apache.org/jira/browse/HDDS-10354> Are nice to have, but not required. I am not planning on taking them up right now but if there are volunteers interested, the current committed CI pipeline is complete enough to plug them in pretty easily* - *Social media preview is fixed* I fixed some bugs related to the social media preview on the site and our new social card, tagline, and page titles should render when a link to the staging site is shared on social media or messaging apps. Try sending https://ozone-site-v2.staged.apache.org/ to yourself on slack to see how it looks. - *Repo-wide default branch change* There was a recent thread on changing the default branch of the website repo from asf-site to master. That work has completed, and has enabled a PR template to be provided for all new PRs across the repo, including those to the website feature branch. No other impact is expected to the new or existing website. With the finish line in sight for website framework (HDDS-9538 <https://issues.apache.org/jira/browse/HDDS-9538>) and GitHub integration ( HDDS-9601 <https://issues.apache.org/jira/browse/HDDS-9601>), I would like to call out some risk areas I’m seeing moving into the next phase of this project: - Up to date document translations Maintaining accurate translations is a challenge for any website. Docusaurus has support for i18n, but that only gives us the ability to add translations. The burden of ensuring they remain current and accurate is still on us. Crowdin <https://crowdin.com/> may be an option to help us here, but it seems Pulsar already tried that with poor results <https://github.com/apache/pulsar/discussions/17810>. If anyone has ideas for tracking translation efforts please submit them. - Professional looking homepage This is the only visual part of the website where Docusaurus does not provide us much help. I’m not sure we have any current Ozone contributors who are very familiar with design to help us create a homepage, but Will Xiao and myself are trying our best in PR 65 <https://github.com/apache/ozone-site/pull/65>. Please submit your design ideas if you have them. It's always good to have options. - Docs content itself The progress script <https://github.com/apache/ozone-site/blob/HDDS-9225-website-v2/progress.sh> tells us that there are 199 incomplete pages stubbed out on the website currently. I estimate this number will grow to between 250-300 as pages are added and split into smaller ones. We will need some more experienced members of the community to help with some particularly challenging sections like bare metal secure installation, but there are plenty of more accessible pages on the website to fill in as well. Overall I feel the project is steadily progressing and thank you all for your support. Ethan On Wed, Jan 24, 2024 at 5:17 PM Ethan Rose <er...@apache.org> wrote: > Hi all, I just wanted to provide some updates on the new website. I will > try to send out these update emails more regularly in this thread to > summarize progress. > > - We have a staging website deployed at > https://ozone-site-v2.staged.apache.org. This is also linked from the > main website Jira HDDS-9225. This is updated with each commit to the > feature branch and is a good way to check the current progress on the site, > as well as test it on mobile and multiple browsers. > - We have a detailed contributing guide > <https://github.com/apache/ozone-site/blob/HDDS-9225-website-v2/CONTRIBUTING.md> > added to the new website feature branch. Most changes will just be > markdown, but this doc has some more docusaurus specific details for those > who are interested. > - We have a proposed outline of sections and pages for the docs. This can > be viewed on the staging website and is subject to change as we create > documentation. This initial template is to help organize contributions as > we begin writing. > - There are currently 200 pages on the website that need to be written. > We will inevitably end up with more than this as existing pages in the > outline are split to subpages or missing pages are added. Right now each > page links to a parent Jira where a subtask can be created to fill it in. > We are investigating some automation to potentially pre-create Jiras for > these pages to make sure two people are not accidentally working on the > same page. > > At this point the framework is complete enough that we can accept > contributions to pages in the new website, and we are looking for > volunteers who would like to help us with this project. > > Thanks, > > Ethan > > On Thu, Nov 2, 2023 at 6:32 PM Ethan Rose <er...@apache.org> wrote: > >> Hi Ozone devs, >> >> It has been a while since I started this thread and got carried off into >> other issues. However, I have recently been able to shift focus back to >> this improvement effort. The overall website improvement is tracked under >> HDDS-9225, which is split into multiple child tasks. I have begun defining >> work items under those tasks and started implementation, but I would >> appreciate any help in this area! All PRs are going to the >> HDDS-9225-website-v2 >> <https://github.com/apache/ozone-site/tree/HDDS-9225-website-v2> branch >> in the ozone-site repo, and can be tagged with the website-v2 >> <https://github.com/apache/ozone-site/pulls?q=label%3Awebsite-v2+> >> Github label. >> >> Here are the items we are ready to begin work on: >> >> - HDDS-9538 <https://issues.apache.org/jira/browse/HDDS-9538>. Subtasks >> of this Jira are for setting up the Docusuarus framework. >> >> I have two PRs out now to get an initial website template committed: >> https://github.com/apache/ozone-site/pull/45 >> https://github.com/apache/ozone-site/pull/46 >> >> Once these are merged, the remaining subtasks of the framework parent >> Jira can be completed in any order. Feel free to file Jiras under this task >> if there are framework related tasks I have missed. >> >> - HDDS-9601 <https://issues.apache.org/jira/browse/HDDS-9601>. Subtasks >> of this Jira are for Github integration with Docusaurus. These will mostly >> be small Github actions related patches if anyone who has worked on the >> main Ozone CI (or who wants to learn GH actions) is interested in >> contributing to the new website. >> >> - HDDS-9539 <https://issues.apache.org/jira/browse/HDDS-9539>. Subtasks >> of this Jira are for creating the website homepage. >> >> Help finding/generating images would be appreciated. This may be easier >> once we have the homepage content better defined. >> >> - HDDS-9613 <https://issues.apache.org/jira/browse/HDDS-9613> >> (documentation) and HDDS-9614 >> <https://issues.apache.org/jira/browse/HDDS-9614>(community) >> >> We are not yet ready to start writing these pages, but I would appreciate >> discussion on the Jiras about the layout and content of pages we should >> include in these sections. >> >> Thanks for your support of this effort, and please let me know if you >> have any questions or concerns about the project. >> >> Ethan >> >> On Wed, Aug 30, 2023 at 9:59 AM Wei-Chiu Chuang <weic...@apache.org> >> wrote: >> >>> In addition to layout revamp, one thing I'd love to see is more user >>> docs. >>> Our doc is by and large written for core Ozone developers. We need to >>> have >>> more content around how to run workarounds, how to migrate existing >>> applications from HDFS to Ozone, API guide ... etc. >>> >>> On Mon, Aug 28, 2023 at 5:25 PM Ethan Rose <er...@cloudera.com.invalid> >>> wrote: >>> >>> > Hello Ozone Community, >>> > >>> > As many of you are aware, Ozone's website and documentation has >>> remained >>> > somewhat disorganized and lacking in content for a while. I would like >>> to >>> > kick start an initiative to change that. I have attached a document >>> > outlining a plan to redo Ozone's website and documentation so that it >>> will >>> > not be a barrier for new adopters and can be easily updated to hold all >>> > content we wish to publish. The document outlines a high level plan >>> that >>> > will be broken into multiple parent jiras with subtasks to follow >>> while the >>> > new website is under construction. I would encourage everyone to read >>> it >>> > and give feedback on this initiative. I also presented this content in >>> this >>> > week's U.S time zone community sync. >>> > >>> > As part of "Phase 1" in the document, I would like to make the >>> following >>> > proposals: >>> > >>> > 1. Migrate Ozone's static site generator from Hugo to Docusaurus. >>> > >>> > Docusaurus <https://docusaurus.io/> is an MIT licensed static site >>> > generator maintained by Meta/Facebook open source. While there are many >>> > examples <https://docusaurus.io/showcase> of sites using Docusaurus, >>> > including the Docusaurus site itself, YuniKorn >>> > <https://yunikorn.apache.org/> is another Apache project that is >>> > successfully using the framework for their website. I have discussed >>> this >>> > with two Yunikorn PMC members who work on the site and they said the >>> > experience has been positive. >>> > >>> > What does Docusaurus provide us that Hugo does not? Hugo's advantage is >>> > that it is written in Go, so in theory the site can be built from just >>> one >>> > easy to install binary. Docusaurus is written in Javascript, so it >>> needs >>> > to be installed from a package manager like npm. However, if we want to >>> > make the site more robust with things like collapsable menus, minimap >>> side >>> > bar, header linking, search, breadcrumbs, and more, we would probably >>> need >>> > to depend on a Hugo theme rather than build all these things from >>> scratch. >>> > Hugo themes with these features, like Doks <https://getdoks.org>, are >>> > also javascript packages that require node to build, but have the >>> following >>> > disadvantages: >>> > - Configuration becomes split between the theme and Hugo itself. It is >>> not >>> > intuitive for new developers whether a config belongs to Hugo or the >>> theme, >>> > or whose documentation to read to make changes. >>> > - The site is now highly dependent on another library that may not get >>> the >>> > same amount of updates or support as the main Hugo project. >>> > >>> > Since Docusaurus is built specifically for project documentation, it >>> > provides all these features with its default theme, and configuration >>> > happens in a centralized docusaurus.config.js file. I have tested it >>> and >>> > can confirm that it is just as easy to spin up the site locally and >>> view >>> > edits in real time as Hugo. It also has a built in versioning >>> > <https://docusaurus.io/docs/versioning> framework that we can use >>> instead >>> > of our manual process of copying over docs from the main Ozone repo, >>> which >>> > brings me to my next proposal. >>> > >>> > 2. Move our docs out of the ozone repo and into the ozone-site repo. >>> > >>> > In theory, having docs tracked and versioned in the same repo as the >>> code >>> > sounds like a good idea, but in practice, we do not use any of the >>> > potential benefits: >>> > - We always use separate PRs to update docs and code, and I think this >>> > actually makes reviews easier. >>> > - The existence of the docs in the same repo as the code does not seem >>> to >>> > "remind" anyone to update the docs if something relevant changes. Zita >>> > suggested at the community sync that adding a reminder to update docs >>> to >>> > our PR template would probably be more effective. >>> > - Relations between code and doc changes can already be created with >>> > existing Jira and Github linking features, even across repos. >>> > >>> > This split model does cause us problems: >>> > - It is challenging to automatically publish incremental updates to the >>> > versioned portion of the docs. In order to update the versioned docs, >>> we >>> > need to build the docs within Ozone, and manually copy the built >>> artifacts >>> > into the asf-site branch of the ozone-site repo. This is a confusing >>> > process, since most doc updates to ozone-site are supposed to happen >>> on the >>> > master branch which is automatically built into the asf-site branch. >>> > Because of this, content updates to our docs are rarely published >>> outside >>> > of a release, even if they are committed earlier. >>> > - The most up to date docs are built from the ozone master branch and >>> must >>> > live on a separate site >>> > < >>> https://ci-hadoop.apache.org/view/Hadoop%20Ozone/job/ozone-doc-master/lastSuccessfulBuild/artifact/hadoop-hdds/docs/public/index.html >>> > >>> > from ozone.apache.org because there is no integration between the main >>> > ozone repo and the automatic doc publishing in the ozone-site repo. >>> > - We must manually version our documentation instead of letting our >>> site >>> > generator do it for us. This can be error prone. For example, see the >>> > comments on the 1.3.0 doc update PR >>> > <https://github.com/apache/ozone-site/pull/28>. >>> > >>> > For these reasons, I propose moving the docs out of the main Ozone repo >>> > and into the ozone-site repo. Work on the improved website and docs can >>> > happen on a branch in the ozone-site repo until it is ready to be >>> published. >>> > >>> > The attached document outlines a detailed and incremental plan to push >>> > Ozone's website and documentation to its full potential. Please review >>> the >>> > plan and let me know your thoughts. I plan for this to be my primary >>> > project for the next few months, with the goal of getting the new site >>> to a >>> > publishable state by the end of this year. A few other engineers at >>> > Cloudera have agreed to help part time, but we would greatly appreciate >>> > help from the open source community of developers and users as we work >>> on >>> > this. Even if you do not have the time or ability to contribute >>> content, >>> > reviews from a wide audience will improve the quality of the new site >>> and >>> > docs. >>> > >>> > Thanks, >>> > Ethan >>> > >>> > --------------------------------------------------------------------- >>> > To unsubscribe, e-mail: dev-unsubscr...@ozone.apache.org >>> > For additional commands, e-mail: dev-h...@ozone.apache.org >>> >>
