Nathan just commented on the Jira about an obvious first step being to restrict HTTP to localhost only by default. This is an easy and immediately doable first step. I am planning to wait for the RC4 creation for that PR to land and get merged.
Thanks On Wed, Feb 10, 2021 at 10:24 AM Nathan Gough <[email protected]> wrote: > I 100% agree that something needs to be done. We cannot allow NiFi to build > a reputation that it is 'insecure' by allowing its default installation to > start up without any security. Especially considering how much work we put > in to make sure it IS a secure product that integrates with many > applications in a secure way. Security reputation is very important for > software. If some major exploitation of NiFi were to happen in the future, > we should be able to confidently say that we did our absolute best to > create a secure application. We shouldn't point at new users and say 'they > didn't configure it right'. > > Personally, I am in favor of providing automatically generated certificates > and requiring the user to insert the client certificate in their browser, > and providing instructions and perhaps a YouTube video on how to do that. > Yes, X509 certificate errors are confusing and can be difficult for > beginners to troubleshoot. But those beginner users will also be the most > likely to use NiFi insecurely, connect it to the internet, and become part > of a user base who got burnt by NiFi being 'insecure'. I acknowledge this > is increasing the barrier for entry. If we intend to use a > username/password + server cert for HTTPs but no client cert, as stated > above we could automatically generate the password and provide this to the > user in a log or file. > > > On Wed, Feb 10, 2021 at 12:21 PM David Handermann < > [email protected]> wrote: > > > Integrating an option to use Let's Encrypt would be a great improvement > > over self-signed certificates, but it is difficult to support that for > > instances of NiFi running on servers without Internet access. Even when > > running NiFi for local development purposes, the development system is > not > > likely to have a publicly addressable DNS name, so Let's Encrypt is not > an > > option in those cases. > > > > Regards, > > David Handermann > > > > On Wed, Feb 10, 2021 at 11:09 AM Joe Witt <[email protected]> wrote: > > > > > Otto > > > > > > Installers like you mention are inherently brutal for portability so > very > > > difficult for us in the community. From the vendor world we of course > do > > > things like that. I think in apache nifi land we need a default 'even > > for > > > devs' which is not wide open. > > > > > > James > > > > > > I do think adding such a warning is good. But it doesn't help avoid > > these > > > wildly abusive cases. We need a secure by default path. We can > probably > > > do better than the self signed path if we add a 'before running' step > in > > > the CLI for the user. Not sure. > > > > > > Thanks > > > > > > On Wed, Feb 10, 2021 at 10:05 AM James Srinivasan < > > > [email protected]> wrote: > > > > > > > Would a suitably large warning on the http ui be a good starting > point? > > > > > > > > Browsers are getting increasingly wary of self signed certs and we > > > probably > > > > don't want to be encouraging people to ignore them. > > > > > > > > What about easier acme+certbot support? (notwithstanding all the non > > > public > > > > deployments) > > > > > > > > On Wed, 10 Feb 2021, 15:25 Otto Fowler, <[email protected]> > > wrote: > > > > > > > > > Aren’t most of these applications installed by an installer? > > > > > Maybe we can have one or more installers that setup a secure > > instance, > > > > and > > > > > those installers > > > > > could be the *production* nifi, and keep the zip distribution open > > for > > > > > developers? > > > > > > > > > > > > > > > > On Feb 10, 2021, at 10:04, David Handermann < > > > > [email protected]> > > > > > wrote: > > > > > > > > > > > > I agree that a generated password is the way to go, and the > > challenge > > > > is > > > > > > making it available to the user. Depending on how it is stored > for > > > the > > > > > > identity provider, having a command line tool to read and print > it > > > > could > > > > > be > > > > > > a reasonable option. > > > > > > > > > > > > Although this particular thread referenced a specific Twitter > post, > > > > this > > > > > > general discussion is more of a reflection of the need to make > > things > > > > > more > > > > > > secure by default as other products have followed similar > > approaches. > > > > > > > > > > > > Regards, > > > > > > David Handermann > > > > > > > > > > > > On Wed, Feb 10, 2021 at 8:53 AM Kevin Doran <[email protected]> > > > wrote: > > > > > > > > > > > >> I am in favor of requiring some authentication by default. > > > > > >> > > > > > >> I’m in favor of an admin username and generated password. (It > > sounds > > > > > li9ke > > > > > >> most of us are on the same page, but I don’t think a static, > > default > > > > > >> password buys us much against the types of abuse shown on the > > > twitter > > > > > >> thread Joe shared.) > > > > > >> > > > > > >> We need some way of making the generated password discoverable… > > > Print > > > > to > > > > > >> the logs on first boot? Not great but are there other > mechanisms? > > A > > > > > setup > > > > > >> CLI utility? > > > > > >> > > > > > >> To help not impede automated deployments, maybe we should change > > the > > > > > >> startup flow such that there is a way to provide this password. > > That > > > > > would > > > > > >> be better than people just disabling whatever default > > authentication > > > > we > > > > > set. > > > > > >> > > > > > >> > > > > > >>> On Feb 10, 2021, at 09:45, David Handermann < > > > > > [email protected]> > > > > > >> wrote: > > > > > >>> > > > > > >>> Mark, > > > > > >>> > > > > > >>> Thanks for clarifying, that makes sense. > > > > > >>> > > > > > >>> Regards, > > > > > >>> David Handermann > > > > > >>> > > > > > >>> On Wed, Feb 10, 2021 at 8:41 AM Mark Payne < > [email protected] > > > > > > > > wrote: > > > > > >>> > > > > > >>>> David, > > > > > >>>> > > > > > >>>> My concern was purely around generating client certs and using > > > > mutual > > > > > >> TLS. > > > > > >>>> I definitely think we should have a server cert if using > > username > > > & > > > > > >>>> password. > > > > > >>>> > > > > > >>>> Thanks > > > > > >>>> -Mark > > > > > >>>> > > > > > >>>> > > > > > >>>>> On Feb 10, 2021, at 9:37 AM, David Handermann < > > > > > >>>> [email protected]> wrote: > > > > > >>>>> > > > > > >>>>> Mark, > > > > > >>>>> > > > > > >>>>> Thanks for your perspective on certificate generation, I > agree > > > that > > > > > >>>>> troubleshooting certificates can be confusing due to unclear > > > error > > > > > >>>>> messaging. Generating self-signed certificates for one-way > TLS > > > > still > > > > > >>>>> requires the user to accept the certificate presented, but at > > > least > > > > > >> that > > > > > >>>> is > > > > > >>>>> more common in various products. Are you concerned about > that > > > > > >> situation, > > > > > >>>>> or are you more concerned about the additional challenges of > > > mutual > > > > > TLS > > > > > >>>>> authentication? > > > > > >>>>> > > > > > >>>>> Generating a random password in absence of any other > > > configuration > > > > > >> would > > > > > >>>>> certainly be easier from a new user perspective. Some of the > > > > > security > > > > > >>>>> concerns with password authentication can be mitigated with > > > one-way > > > > > >> TLS, > > > > > >>>> so > > > > > >>>>> a blending of these approaches, as Joe describes in > NIFI-8220, > > > > seems > > > > > >>>> like a > > > > > >>>>> good way to go. > > > > > >>>>> > > > > > >>>>> Regards, > > > > > >>>>> David Handermann > > > > > >>>>> > > > > > >>>>> On Wed, Feb 10, 2021 at 8:23 AM Mark Payne < > > [email protected] > > > > > > > > > >> wrote: > > > > > >>>>> > > > > > >>>>>> I would be in favor of this as well. I agree that there is > no > > > need > > > > > to > > > > > >>>> wait > > > > > >>>>>> for a 2.0 version - there is no inherit backward > compatibility > > > > > >> challenge > > > > > >>>>>> here. > > > > > >>>>>> Requiring that new application configs be set, etc. I think > is > > > > > >>>> completely > > > > > >>>>>> fair game between minor versions. > > > > > >>>>>> > > > > > >>>>>> Personally, though, I would very much stray away from > > > > > auto-generating > > > > > >>>>>> certificates. For those of us who have dealt with > certificates > > > > > >>>> significantly > > > > > >>>>>> In the past, minor configuration issues are easy to address. > > But > > > > for > > > > > >>>>>> someone who hasn’t spent much time dealing with > certificates, > > > the > > > > > >> error > > > > > >>>>>> messages > > > > > >>>>>> that are often intentionally vague can quickly result in > users > > > > being > > > > > >>>>>> overwhelmed. This is especially true if browser > configurations > > > are > > > > > >>>> already > > > > > >>>>>> setup to > > > > > >>>>>> automatically offer certificates that area already > installed - > > > > this > > > > > >> can > > > > > >>>>>> result in weird error messages about untrusted certificates > > when > > > > the > > > > > >>>> user > > > > > >>>>>> thinks > > > > > >>>>>> they haven’t even provided a certificate, etc. It gets > really > > > > hairy. > > > > > >>>>>> > > > > > >>>>>> I am more in favor of a default username/password > personally. > > It > > > > > would > > > > > >>>>>> require implementing a new auth provider. And it’s one that > > > > > >>>> historically we > > > > > >>>>>> have > > > > > >>>>>> avoided implementing because a basic auth type of approach > is > > > less > > > > > >>>> secure > > > > > >>>>>> than mutual auth, ldap, etc. But it’s more secure than > > nothing. > > > > > >>>>>> > > > > > >>>>>> Thanks > > > > > >>>>>> -Mark > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>>>>> On Feb 10, 2021, at 9:13 AM, Joe Witt <[email protected]> > > > > wrote: > > > > > >>>>>>> > > > > > >>>>>>> There are a range of things to consider. And yes whatever > we > > > do > > > > > will > > > > > >>>>>>> involve writing code. We also have to find the right place > > for > > > > the > > > > > >> bar > > > > > >>>>>>> here. > > > > > >>>>>>> > > > > > >>>>>>> 1. Disable HTTP by default. And if they want to enable > HTTP > > > they > > > > > >>>> should > > > > > >>>>>>> also have to make a config change indicating they are fully > > > > willing > > > > > >> to > > > > > >>>>>>> accept that it is an entirely non secure configuration as > far > > > as > > > > > the > > > > > >>>>>>> application is concerned. > > > > > >>>>>>> 2. Provide a default username/password provider out of the > > box > > > > > >>>> configured > > > > > >>>>>>> by default and an auto generate self-signed server cert. > > This > > > > > means > > > > > >>>> the > > > > > >>>>>>> NiFi server will provide the client browser a cert. It > won't > > > be > > > > > >>>>>>> known/trusted so the browser will advise of this. And on > > > startup > > > > > >> NiFi > > > > > >>>>>> can > > > > > >>>>>>> auto generate and log this username and password. It would > > > truly > > > > > be > > > > > >> a > > > > > >>>>>>> single username/password and not some store for various > > users. > > > > We > > > > > >>>>>>> disallow access to all restricted components by default. > > > > > >>>>>>> > > > > > >>>>>>> This default configuration at least means someone cannot > > start > > > > NiFi > > > > > >> and > > > > > >>>>>> it > > > > > >>>>>>> is totally exposed by default while also preserving a > pretty > > > > simple > > > > > >>>> 'get > > > > > >>>>>>> started' model for the user. They'd have to take action to > > > make > > > > it > > > > > >>>> less > > > > > >>>>>>> secure. > > > > > >>>>>>> > > > > > >>>>>>> The option DavidH mentions of mutual auth could work well > > also > > > > and > > > > > as > > > > > >>>> he > > > > > >>>>>>> mentioned avoids the need for anything special in terms of > > auth > > > > > >>>> provider > > > > > >>>>>>> which is compelling for us but I do worry about the user > > > > experience > > > > > >> on > > > > > >>>>>> that. > > > > > >>>>>>> > > > > > >>>>>>> I'll add to this that I think we cannot afford to wait for > a > > > NiFi > > > > > 2.0 > > > > > >>>>>>> release to take action here. Or that we should expedite a > > NiFi > > > > 2.0 > > > > > >>>>>> release > > > > > >>>>>>> to take action and that we should just not make the bar for > > > what > > > > > 2.0 > > > > > >> is > > > > > >>>>>> so > > > > > >>>>>>> high that we never get this done. But frankly I think we > > could > > > > > make > > > > > >>>> this > > > > > >>>>>>> change in NiFi 1.15 and document what is happening. For > > > existing > > > > > >>>>>>> installs/configs we'd not be changing anything except maybe > > > when > > > > > >>>> they're > > > > > >>>>>>> using HTTP and don't set the 'no seriously i want this > thing > > > wide > > > > > >> open > > > > > >>>>>>> option'. > > > > > >>>>>>> > > > > > >>>>>>> Thanks > > > > > >>>>>>> > > > > > >>>>>>> On Wed, Feb 10, 2021 at 6:58 AM David Handermann < > > > > > >>>>>> [email protected]> > > > > > >>>>>>> wrote: > > > > > >>>>>>> > > > > > >>>>>>>> Having NiFi enforce authentication by default seems like > the > > > > right > > > > > >> way > > > > > >>>>>> to > > > > > >>>>>>>> go, especially given the capabilities of the system. > > > > > >>>>>>>> > > > > > >>>>>>>> Bryan raises a good point about storage of account > > > information, > > > > so > > > > > >>>>>> weighing > > > > > >>>>>>>> the positives and negatives of various identity providers > > > should > > > > > be > > > > > >>>>>>>> considered. > > > > > >>>>>>>> > > > > > >>>>>>>> Following on Joe's point about disabling plain HTTP, one > > > option > > > > > >> could > > > > > >>>> be > > > > > >>>>>>>> generating both client and server certificates and using > > > Mutual > > > > > TLS > > > > > >>>> for > > > > > >>>>>>>> authentication. This would obviously require installing > the > > > > > client > > > > > >>>>>>>> certificate in a browser, which is not trivial, but could > be > > > > part > > > > > of > > > > > >>>> an > > > > > >>>>>>>> installation guide. This approach definitely provides a > > high > > > > > >> barrier > > > > > >>>> of > > > > > >>>>>>>> entry of new users, but provides a strong level of > security > > > > while > > > > > >>>>>> avoiding > > > > > >>>>>>>> the need for some other identity provider service to be > > > > > configured. > > > > > >>>> As > > > > > >>>>>>>> others have mentioned, this seems necessary to address the > > > > > situation > > > > > >>>> of > > > > > >>>>>>>> someone installing NiFi without a full understanding of > the > > > > > >>>>>> configuration > > > > > >>>>>>>> required, so it is important to keep the audience in mind > > when > > > > > >>>>>> evaluating a > > > > > >>>>>>>> solution. > > > > > >>>>>>>> > > > > > >>>>>>>> Regards, > > > > > >>>>>>>> David Handermann > > > > > >>>>>>>> > > > > > >>>>>>>> On Wed, Feb 10, 2021 at 7:39 AM Bryan Bende < > > [email protected] > > > > > > > > > >> wrote: > > > > > >>>>>>>> > > > > > >>>>>>>>> Just to clarify, I was not suggesting that we make a > > default > > > > > secure > > > > > >>>>>>>>> setup that requires LDAP. > > > > > >>>>>>>>> > > > > > >>>>>>>>> I was just saying that in order to provide a default > secure > > > > > setup, > > > > > >>>>>>>>> we'd have to provide a login identity provider > > implementation > > > > > that > > > > > >> is > > > > > >>>>>>>>> backed by a file or database table, which in the past we > > > > decided > > > > > >>>>>>>>> against. > > > > > >>>>>>>>> > > > > > >>>>>>>>> On Wed, Feb 10, 2021 at 8:28 AM Russell Bateman < > > > > > >>>> [email protected] > > > > > >>>>>>> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> I second the concerns expressed, but second especially > > > Bryan's > > > > > >>>>>> pointing > > > > > >>>>>>>>>> out that requiring LDAP/AD to be set up in order even to > > > begin > > > > > to > > > > > >>>> use > > > > > >>>>>>>>>> our framework would be a bit onerous for developers just > > > > > >> interested > > > > > >>>> in > > > > > >>>>>>>>>> getting work done and a barrier to considering the > > framework > > > > > >> should > > > > > >>>> it > > > > > >>>>>>>>>> be erected a little too high. Should we at least glance > at > > > how > > > > > >> this > > > > > >>>> is > > > > > >>>>>>>>>> solved by the likes of other projects, Kafka and > Cassandra > > > > come > > > > > to > > > > > >>>>>>>> mind, > > > > > >>>>>>>>>> even if it means resorting to a store of a name or two? > I > > > > didn't > > > > > >>>> find > > > > > >>>>>>>>>> getting into developing with them a pain, but making me > > jump > > > > > >> through > > > > > >>>>>>>> the > > > > > >>>>>>>>>> hoop of setting up LDAP may very well have changed that. > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> These unsecure instances of NiFi out there are not our > > > > > community's > > > > > >>>>>>>>>> fault. I suppose we're worried about getting splattered > by > > > bad > > > > > >>>> press? > > > > > >>>>>>>>>> > > > > > >>>>>>>>>> On 2/10/21 5:47 AM, Bryan Bende wrote: > > > > > >>>>>>>>>>> I agree with the overall idea, although I would think > it > > > > > >> requires a > > > > > >>>>>>>>>>> major release to make this kind of change to the > default > > > > > >> behavior. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Also, we have always avoided NiFi being a store of > > > usernames > > > > > and > > > > > >>>>>>>>>>> passwords, so we don't have a login provider that uses > a > > > > local > > > > > >> file > > > > > >>>>>>>> or > > > > > >>>>>>>>>>> a database, we've always said you connect to LDAP/AD > for > > > > that. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> Obviously it can be implemented, but just pointing out > > that > > > > > we'd > > > > > >>>> have > > > > > >>>>>>>>>>> to change our stance here if we want to provide a > default > > > > > >> username > > > > > >>>>>>>> and > > > > > >>>>>>>>>>> password to authenticate with. > > > > > >>>>>>>>>>> > > > > > >>>>>>>>>>> On Tue, Feb 9, 2021 at 11:25 PM Andrew Grande < > > > > > >> [email protected]> > > > > > >>>>>>>>> wrote: > > > > > >>>>>>>>>>>> Mysql has been generating an admin password on default > > > > > installs > > > > > >>>> for, > > > > > >>>>>>>>> like, > > > > > >>>>>>>>>>>> forever. This workflow should be familiar for many > > users. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> I'd suggest taking the automation tooling into account > > and > > > > > how a > > > > > >>>>>>>>> production > > > > > >>>>>>>>>>>> rollout (user-provided password) would fit into the > > > > workflow. > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> Andrew > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>> On Tue, Feb 9, 2021, 8:15 PM Tony Kurc < > > [email protected]> > > > > > >> wrote: > > > > > >>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Joe, > > > > > >>>>>>>>>>>>> In addition to your suggestions, were you thinking of > > > > making > > > > > >> this > > > > > >>>>>>>>> processor > > > > > >>>>>>>>>>>>> disabled by default as well? > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> Tony > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>> On Tue, Feb 9, 2021, 11:04 PM Joe Witt < > > > [email protected] > > > > > > > > > > >>>> wrote: > > > > > >>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Team > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> While secure by default may not be practical perhaps > > > ‘not > > > > > >>>>>>>> blatantly > > > > > >>>>>>>>> wide > > > > > >>>>>>>>>>>>>> open’ by default should be adopted. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> I think we should consider killing support for http > > > > entirely > > > > > >> and > > > > > >>>>>>>>> support > > > > > >>>>>>>>>>>>>> only https. We should consider auto generating a > user > > > and > > > > > >>>>>>>> password > > > > > >>>>>>>>> and > > > > > >>>>>>>>>>>>>> possibly server cert if nothing is configured and > log > > > the > > > > > >>>>>>>> generated > > > > > >>>>>>>>> user > > > > > >>>>>>>>>>>>>> and password. Sure it could still be configured to > > be > > > > non > > > > > >>>> secure > > > > > >>>>>>>>> but > > > > > >>>>>>>>>>>>> that > > > > > >>>>>>>>>>>>>> would truly be an admins fault. Now its just ‘on’ > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> This tweet is a great example of why > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > https://twitter.com/_escctrl_/status/1359280656174510081?s=21 > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Who agrees? Who disagrees? Please share ideas. > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>>>>>>> Thanks > > > > > >>>>>>>>>>>>>> > > > > > >>>>>>>>> > > > > > >>>>>>>> > > > > > >>>>>> > > > > > >>>>>> > > > > > >>>> > > > > > >>>> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > >
