I 100% agree that something needs to be done. We cannot allow NiFi to build a reputation that it is 'insecure' by allowing its default installation to start up without any security. Especially considering how much work we put in to make sure it IS a secure product that integrates with many applications in a secure way. Security reputation is very important for software. If some major exploitation of NiFi were to happen in the future, we should be able to confidently say that we did our absolute best to create a secure application. We shouldn't point at new users and say 'they didn't configure it right'.
Personally, I am in favor of providing automatically generated certificates and requiring the user to insert the client certificate in their browser, and providing instructions and perhaps a YouTube video on how to do that. Yes, X509 certificate errors are confusing and can be difficult for beginners to troubleshoot. But those beginner users will also be the most likely to use NiFi insecurely, connect it to the internet, and become part of a user base who got burnt by NiFi being 'insecure'. I acknowledge this is increasing the barrier for entry. If we intend to use a username/password + server cert for HTTPs but no client cert, as stated above we could automatically generate the password and provide this to the user in a log or file. On Wed, Feb 10, 2021 at 12:21 PM David Handermann < [email protected]> wrote: > Integrating an option to use Let's Encrypt would be a great improvement > over self-signed certificates, but it is difficult to support that for > instances of NiFi running on servers without Internet access. Even when > running NiFi for local development purposes, the development system is not > likely to have a publicly addressable DNS name, so Let's Encrypt is not an > option in those cases. > > Regards, > David Handermann > > On Wed, Feb 10, 2021 at 11:09 AM Joe Witt <[email protected]> wrote: > > > Otto > > > > Installers like you mention are inherently brutal for portability so very > > difficult for us in the community. From the vendor world we of course do > > things like that. I think in apache nifi land we need a default 'even > for > > devs' which is not wide open. > > > > James > > > > I do think adding such a warning is good. But it doesn't help avoid > these > > wildly abusive cases. We need a secure by default path. We can probably > > do better than the self signed path if we add a 'before running' step in > > the CLI for the user. Not sure. > > > > Thanks > > > > On Wed, Feb 10, 2021 at 10:05 AM James Srinivasan < > > [email protected]> wrote: > > > > > Would a suitably large warning on the http ui be a good starting point? > > > > > > Browsers are getting increasingly wary of self signed certs and we > > probably > > > don't want to be encouraging people to ignore them. > > > > > > What about easier acme+certbot support? (notwithstanding all the non > > public > > > deployments) > > > > > > On Wed, 10 Feb 2021, 15:25 Otto Fowler, <[email protected]> > wrote: > > > > > > > Aren’t most of these applications installed by an installer? > > > > Maybe we can have one or more installers that setup a secure > instance, > > > and > > > > those installers > > > > could be the *production* nifi, and keep the zip distribution open > for > > > > developers? > > > > > > > > > > > > > On Feb 10, 2021, at 10:04, David Handermann < > > > [email protected]> > > > > wrote: > > > > > > > > > > I agree that a generated password is the way to go, and the > challenge > > > is > > > > > making it available to the user. Depending on how it is stored for > > the > > > > > identity provider, having a command line tool to read and print it > > > could > > > > be > > > > > a reasonable option. > > > > > > > > > > Although this particular thread referenced a specific Twitter post, > > > this > > > > > general discussion is more of a reflection of the need to make > things > > > > more > > > > > secure by default as other products have followed similar > approaches. > > > > > > > > > > Regards, > > > > > David Handermann > > > > > > > > > > On Wed, Feb 10, 2021 at 8:53 AM Kevin Doran <[email protected]> > > wrote: > > > > > > > > > >> I am in favor of requiring some authentication by default. > > > > >> > > > > >> I’m in favor of an admin username and generated password. (It > sounds > > > > li9ke > > > > >> most of us are on the same page, but I don’t think a static, > default > > > > >> password buys us much against the types of abuse shown on the > > twitter > > > > >> thread Joe shared.) > > > > >> > > > > >> We need some way of making the generated password discoverable… > > Print > > > to > > > > >> the logs on first boot? Not great but are there other mechanisms? > A > > > > setup > > > > >> CLI utility? > > > > >> > > > > >> To help not impede automated deployments, maybe we should change > the > > > > >> startup flow such that there is a way to provide this password. > That > > > > would > > > > >> be better than people just disabling whatever default > authentication > > > we > > > > set. > > > > >> > > > > >> > > > > >>> On Feb 10, 2021, at 09:45, David Handermann < > > > > [email protected]> > > > > >> wrote: > > > > >>> > > > > >>> Mark, > > > > >>> > > > > >>> Thanks for clarifying, that makes sense. > > > > >>> > > > > >>> Regards, > > > > >>> David Handermann > > > > >>> > > > > >>> On Wed, Feb 10, 2021 at 8:41 AM Mark Payne <[email protected] > > > > > > wrote: > > > > >>> > > > > >>>> David, > > > > >>>> > > > > >>>> My concern was purely around generating client certs and using > > > mutual > > > > >> TLS. > > > > >>>> I definitely think we should have a server cert if using > username > > & > > > > >>>> password. > > > > >>>> > > > > >>>> Thanks > > > > >>>> -Mark > > > > >>>> > > > > >>>> > > > > >>>>> On Feb 10, 2021, at 9:37 AM, David Handermann < > > > > >>>> [email protected]> wrote: > > > > >>>>> > > > > >>>>> Mark, > > > > >>>>> > > > > >>>>> Thanks for your perspective on certificate generation, I agree > > that > > > > >>>>> troubleshooting certificates can be confusing due to unclear > > error > > > > >>>>> messaging. Generating self-signed certificates for one-way TLS > > > still > > > > >>>>> requires the user to accept the certificate presented, but at > > least > > > > >> that > > > > >>>> is > > > > >>>>> more common in various products. Are you concerned about that > > > > >> situation, > > > > >>>>> or are you more concerned about the additional challenges of > > mutual > > > > TLS > > > > >>>>> authentication? > > > > >>>>> > > > > >>>>> Generating a random password in absence of any other > > configuration > > > > >> would > > > > >>>>> certainly be easier from a new user perspective. Some of the > > > > security > > > > >>>>> concerns with password authentication can be mitigated with > > one-way > > > > >> TLS, > > > > >>>> so > > > > >>>>> a blending of these approaches, as Joe describes in NIFI-8220, > > > seems > > > > >>>> like a > > > > >>>>> good way to go. > > > > >>>>> > > > > >>>>> Regards, > > > > >>>>> David Handermann > > > > >>>>> > > > > >>>>> On Wed, Feb 10, 2021 at 8:23 AM Mark Payne < > [email protected] > > > > > > > >> wrote: > > > > >>>>> > > > > >>>>>> I would be in favor of this as well. I agree that there is no > > need > > > > to > > > > >>>> wait > > > > >>>>>> for a 2.0 version - there is no inherit backward compatibility > > > > >> challenge > > > > >>>>>> here. > > > > >>>>>> Requiring that new application configs be set, etc. I think is > > > > >>>> completely > > > > >>>>>> fair game between minor versions. > > > > >>>>>> > > > > >>>>>> Personally, though, I would very much stray away from > > > > auto-generating > > > > >>>>>> certificates. For those of us who have dealt with certificates > > > > >>>> significantly > > > > >>>>>> In the past, minor configuration issues are easy to address. > But > > > for > > > > >>>>>> someone who hasn’t spent much time dealing with certificates, > > the > > > > >> error > > > > >>>>>> messages > > > > >>>>>> that are often intentionally vague can quickly result in users > > > being > > > > >>>>>> overwhelmed. This is especially true if browser configurations > > are > > > > >>>> already > > > > >>>>>> setup to > > > > >>>>>> automatically offer certificates that area already installed - > > > this > > > > >> can > > > > >>>>>> result in weird error messages about untrusted certificates > when > > > the > > > > >>>> user > > > > >>>>>> thinks > > > > >>>>>> they haven’t even provided a certificate, etc. It gets really > > > hairy. > > > > >>>>>> > > > > >>>>>> I am more in favor of a default username/password personally. > It > > > > would > > > > >>>>>> require implementing a new auth provider. And it’s one that > > > > >>>> historically we > > > > >>>>>> have > > > > >>>>>> avoided implementing because a basic auth type of approach is > > less > > > > >>>> secure > > > > >>>>>> than mutual auth, ldap, etc. But it’s more secure than > nothing. > > > > >>>>>> > > > > >>>>>> Thanks > > > > >>>>>> -Mark > > > > >>>>>> > > > > >>>>>> > > > > >>>>>>> On Feb 10, 2021, at 9:13 AM, Joe Witt <[email protected]> > > > wrote: > > > > >>>>>>> > > > > >>>>>>> There are a range of things to consider. And yes whatever we > > do > > > > will > > > > >>>>>>> involve writing code. We also have to find the right place > for > > > the > > > > >> bar > > > > >>>>>>> here. > > > > >>>>>>> > > > > >>>>>>> 1. Disable HTTP by default. And if they want to enable HTTP > > they > > > > >>>> should > > > > >>>>>>> also have to make a config change indicating they are fully > > > willing > > > > >> to > > > > >>>>>>> accept that it is an entirely non secure configuration as far > > as > > > > the > > > > >>>>>>> application is concerned. > > > > >>>>>>> 2. Provide a default username/password provider out of the > box > > > > >>>> configured > > > > >>>>>>> by default and an auto generate self-signed server cert. > This > > > > means > > > > >>>> the > > > > >>>>>>> NiFi server will provide the client browser a cert. It won't > > be > > > > >>>>>>> known/trusted so the browser will advise of this. And on > > startup > > > > >> NiFi > > > > >>>>>> can > > > > >>>>>>> auto generate and log this username and password. It would > > truly > > > > be > > > > >> a > > > > >>>>>>> single username/password and not some store for various > users. > > > We > > > > >>>>>>> disallow access to all restricted components by default. > > > > >>>>>>> > > > > >>>>>>> This default configuration at least means someone cannot > start > > > NiFi > > > > >> and > > > > >>>>>> it > > > > >>>>>>> is totally exposed by default while also preserving a pretty > > > simple > > > > >>>> 'get > > > > >>>>>>> started' model for the user. They'd have to take action to > > make > > > it > > > > >>>> less > > > > >>>>>>> secure. > > > > >>>>>>> > > > > >>>>>>> The option DavidH mentions of mutual auth could work well > also > > > and > > > > as > > > > >>>> he > > > > >>>>>>> mentioned avoids the need for anything special in terms of > auth > > > > >>>> provider > > > > >>>>>>> which is compelling for us but I do worry about the user > > > experience > > > > >> on > > > > >>>>>> that. > > > > >>>>>>> > > > > >>>>>>> I'll add to this that I think we cannot afford to wait for a > > NiFi > > > > 2.0 > > > > >>>>>>> release to take action here. Or that we should expedite a > NiFi > > > 2.0 > > > > >>>>>> release > > > > >>>>>>> to take action and that we should just not make the bar for > > what > > > > 2.0 > > > > >> is > > > > >>>>>> so > > > > >>>>>>> high that we never get this done. But frankly I think we > could > > > > make > > > > >>>> this > > > > >>>>>>> change in NiFi 1.15 and document what is happening. For > > existing > > > > >>>>>>> installs/configs we'd not be changing anything except maybe > > when > > > > >>>> they're > > > > >>>>>>> using HTTP and don't set the 'no seriously i want this thing > > wide > > > > >> open > > > > >>>>>>> option'. > > > > >>>>>>> > > > > >>>>>>> Thanks > > > > >>>>>>> > > > > >>>>>>> On Wed, Feb 10, 2021 at 6:58 AM David Handermann < > > > > >>>>>> [email protected]> > > > > >>>>>>> wrote: > > > > >>>>>>> > > > > >>>>>>>> Having NiFi enforce authentication by default seems like the > > > right > > > > >> way > > > > >>>>>> to > > > > >>>>>>>> go, especially given the capabilities of the system. > > > > >>>>>>>> > > > > >>>>>>>> Bryan raises a good point about storage of account > > information, > > > so > > > > >>>>>> weighing > > > > >>>>>>>> the positives and negatives of various identity providers > > should > > > > be > > > > >>>>>>>> considered. > > > > >>>>>>>> > > > > >>>>>>>> Following on Joe's point about disabling plain HTTP, one > > option > > > > >> could > > > > >>>> be > > > > >>>>>>>> generating both client and server certificates and using > > Mutual > > > > TLS > > > > >>>> for > > > > >>>>>>>> authentication. This would obviously require installing the > > > > client > > > > >>>>>>>> certificate in a browser, which is not trivial, but could be > > > part > > > > of > > > > >>>> an > > > > >>>>>>>> installation guide. This approach definitely provides a > high > > > > >> barrier > > > > >>>> of > > > > >>>>>>>> entry of new users, but provides a strong level of security > > > while > > > > >>>>>> avoiding > > > > >>>>>>>> the need for some other identity provider service to be > > > > configured. > > > > >>>> As > > > > >>>>>>>> others have mentioned, this seems necessary to address the > > > > situation > > > > >>>> of > > > > >>>>>>>> someone installing NiFi without a full understanding of the > > > > >>>>>> configuration > > > > >>>>>>>> required, so it is important to keep the audience in mind > when > > > > >>>>>> evaluating a > > > > >>>>>>>> solution. > > > > >>>>>>>> > > > > >>>>>>>> Regards, > > > > >>>>>>>> David Handermann > > > > >>>>>>>> > > > > >>>>>>>> On Wed, Feb 10, 2021 at 7:39 AM Bryan Bende < > [email protected] > > > > > > > >> wrote: > > > > >>>>>>>> > > > > >>>>>>>>> Just to clarify, I was not suggesting that we make a > default > > > > secure > > > > >>>>>>>>> setup that requires LDAP. > > > > >>>>>>>>> > > > > >>>>>>>>> I was just saying that in order to provide a default secure > > > > setup, > > > > >>>>>>>>> we'd have to provide a login identity provider > implementation > > > > that > > > > >> is > > > > >>>>>>>>> backed by a file or database table, which in the past we > > > decided > > > > >>>>>>>>> against. > > > > >>>>>>>>> > > > > >>>>>>>>> On Wed, Feb 10, 2021 at 8:28 AM Russell Bateman < > > > > >>>> [email protected] > > > > >>>>>>> > > > > >>>>>>>>> wrote: > > > > >>>>>>>>>> > > > > >>>>>>>>>> I second the concerns expressed, but second especially > > Bryan's > > > > >>>>>> pointing > > > > >>>>>>>>>> out that requiring LDAP/AD to be set up in order even to > > begin > > > > to > > > > >>>> use > > > > >>>>>>>>>> our framework would be a bit onerous for developers just > > > > >> interested > > > > >>>> in > > > > >>>>>>>>>> getting work done and a barrier to considering the > framework > > > > >> should > > > > >>>> it > > > > >>>>>>>>>> be erected a little too high. Should we at least glance at > > how > > > > >> this > > > > >>>> is > > > > >>>>>>>>>> solved by the likes of other projects, Kafka and Cassandra > > > come > > > > to > > > > >>>>>>>> mind, > > > > >>>>>>>>>> even if it means resorting to a store of a name or two? I > > > didn't > > > > >>>> find > > > > >>>>>>>>>> getting into developing with them a pain, but making me > jump > > > > >> through > > > > >>>>>>>> the > > > > >>>>>>>>>> hoop of setting up LDAP may very well have changed that. > > > > >>>>>>>>>> > > > > >>>>>>>>>> These unsecure instances of NiFi out there are not our > > > > community's > > > > >>>>>>>>>> fault. I suppose we're worried about getting splattered by > > bad > > > > >>>> press? > > > > >>>>>>>>>> > > > > >>>>>>>>>> On 2/10/21 5:47 AM, Bryan Bende wrote: > > > > >>>>>>>>>>> I agree with the overall idea, although I would think it > > > > >> requires a > > > > >>>>>>>>>>> major release to make this kind of change to the default > > > > >> behavior. > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> Also, we have always avoided NiFi being a store of > > usernames > > > > and > > > > >>>>>>>>>>> passwords, so we don't have a login provider that uses a > > > local > > > > >> file > > > > >>>>>>>> or > > > > >>>>>>>>>>> a database, we've always said you connect to LDAP/AD for > > > that. > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> Obviously it can be implemented, but just pointing out > that > > > > we'd > > > > >>>> have > > > > >>>>>>>>>>> to change our stance here if we want to provide a default > > > > >> username > > > > >>>>>>>> and > > > > >>>>>>>>>>> password to authenticate with. > > > > >>>>>>>>>>> > > > > >>>>>>>>>>> On Tue, Feb 9, 2021 at 11:25 PM Andrew Grande < > > > > >> [email protected]> > > > > >>>>>>>>> wrote: > > > > >>>>>>>>>>>> Mysql has been generating an admin password on default > > > > installs > > > > >>>> for, > > > > >>>>>>>>> like, > > > > >>>>>>>>>>>> forever. This workflow should be familiar for many > users. > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> I'd suggest taking the automation tooling into account > and > > > > how a > > > > >>>>>>>>> production > > > > >>>>>>>>>>>> rollout (user-provided password) would fit into the > > > workflow. > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> Andrew > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>> On Tue, Feb 9, 2021, 8:15 PM Tony Kurc < > [email protected]> > > > > >> wrote: > > > > >>>>>>>>>>>> > > > > >>>>>>>>>>>>> Joe, > > > > >>>>>>>>>>>>> In addition to your suggestions, were you thinking of > > > making > > > > >> this > > > > >>>>>>>>> processor > > > > >>>>>>>>>>>>> disabled by default as well? > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> Tony > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>> On Tue, Feb 9, 2021, 11:04 PM Joe Witt < > > [email protected] > > > > > > > > >>>> wrote: > > > > >>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Team > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> While secure by default may not be practical perhaps > > ‘not > > > > >>>>>>>> blatantly > > > > >>>>>>>>> wide > > > > >>>>>>>>>>>>>> open’ by default should be adopted. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> I think we should consider killing support for http > > > entirely > > > > >> and > > > > >>>>>>>>> support > > > > >>>>>>>>>>>>>> only https. We should consider auto generating a user > > and > > > > >>>>>>>> password > > > > >>>>>>>>> and > > > > >>>>>>>>>>>>>> possibly server cert if nothing is configured and log > > the > > > > >>>>>>>> generated > > > > >>>>>>>>> user > > > > >>>>>>>>>>>>>> and password. Sure it could still be configured to > be > > > non > > > > >>>> secure > > > > >>>>>>>>> but > > > > >>>>>>>>>>>>> that > > > > >>>>>>>>>>>>>> would truly be an admins fault. Now its just ‘on’ > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> This tweet is a great example of why > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> > > > > https://twitter.com/_escctrl_/status/1359280656174510081?s=21 > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Who agrees? Who disagrees? Please share ideas. > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>>>>>>> Thanks > > > > >>>>>>>>>>>>>> > > > > >>>>>>>>> > > > > >>>>>>>> > > > > >>>>>> > > > > >>>>>> > > > > >>>> > > > > >>>> > > > > >> > > > > >> > > > > > > > > > > > > > >
