Hi Szabolcs, I am testing this scenario with our client's slave database. And I am sorry that I can not share the table definition and the sample data here. But attached is a sample of table definition with the column types.
It's quite complex. Thanks, Wenxing On Wed, Jan 4, 2017 at 4:24 PM, Szabolcs Vasas <va...@cloudera.com> wrote: > Hi Wenxing, > > I haven't tried this scenario yet but I would be happy to test it on my > side. Can you please send me the DDL statement for creating the MySQL table > and some sample data? > Also it would be very helpful to send the details of the job you would like > to run. > > Regards, > Szabolcs > > On Wed, Jan 4, 2017 at 2:54 AM, wenxing zheng <wenxing.zh...@gmail.com> > wrote: > > > can anyone help to advice? > > > > And I met with a problem when I set the checkColumn with updated_time, > but > > currently all the updated_time are in NULL. Under this case, the Sqoop > will > > fail to start the job. I think we need to support such kind of case. > > > > On Thu, Dec 29, 2016 at 9:18 AM, wenxing zheng <wenxing.zh...@gmail.com> > > wrote: > > > > > Dear all, > > > > > > Did anyone already try to import more than 10 million data from MySQL > to > > > HDFS by using the Sqoop2? > > > > > > I always failed at the very beginning with various throttling settings, > > > but never made it. > > > > > > Appreciated for any advice. > > > Thanks, Wenxing > > > > > > > > > -- > Szabolcs Vasas > Software Engineer > <http://www.cloudera.com> >
nt(11) NOT NULL AUTO_INCREMENT d` varchar(30) DEFAULT NULL , der_res_id` varchar(300) DEFAUL ic_id` varchar(1000) DEFAULT NU _family` int(11) DEFAULT NULL , _type` int(11) DEFAULT NULL , se_type` int(11) DEFAULT NULL , ical_style` int(11) DEFAULT NUL _format` varchar(10) DEFAULT NU ` text , on` text , ategory` int(11) DEFAULT NULL , ory` varchar(3000) DEFAULT NULL ty_rank` int(11) DEFAULT NULL C der_id` int(11) DEFAULT NULL CO ction_id` int(11) DEFAULT NULL ct_id` int(11) DEFAULT NULL COM rds` varchar(3000) DEFAULT NULL ay_keywords` varchar(3000) DEFA der_group_id` int(11) DEFAULT N group_ids` varchar(500) DEFAULT _state` int(11) DEFAULT NULL CO e_state` int(11) DEFAULT NULL C e_time` datetime DEFAULT NULL C ne_frozen_time` datetime DEFAUL ne_frozen_reason` varchar(100) t_line` varchar(200) DEFAULT NU ` int(11) DEFAULT NULL COMMENT t` int(11) DEFAULT NULL COMMENT int(11) DEFAULT NULL COMMENT ' _type` int(11) DEFAULT NULL COM camera_shot` datetime DEFAULT N ry` varchar(254) DEFAULT NULL, nce` varchar(50) DEFAULT NULL C varchar(254) DEFAULT NULL COMM ion` varchar(200) DEFAULT NULL e` varchar(200) DEFAULT NULL CO t_reason` varchar(100) DEFAULT e_time` datetime DEFAULT NULL C y_time` datetime DEFAULT NULL, d1` varchar(100) DEFAULT NULL , d2` varchar(100) DEFAULT NULL , d3` varchar(100) DEFAULT NULL , d4` varchar(100) DEFAULT NULL , d5` varchar(100) DEFAULT NULL , d6` varchar(100) DEFAULT NULL , d7` varchar(100) DEFAULT NULL , d8` varchar(100) DEFAULT NULL, d9` varchar(100) DEFAULT NULL, d10` varchar(100) DEFAULT NULL, h` varchar(150) DEFAULT NULL , aker` varchar(200) DEFAULT NULL rreto` varchar(100) DEFAULT NUL n` varchar(100) DEFAULT NULL , peed` int(11) DEFAULT NULL , ns` varchar(200) DEFAULT NULL , varchar(50) DEFAULT NULL , time` datetime DEFAULT NULL , user_id` int(11) DEFAULT NULL , stil` int(11) DEFAULT NULL , l_time` datetime DEFAULT NULL , _ids` varchar(3000) DEFAULT NUL pyright` int(11) DEFAULT NULL , ight` varchar(1000) DEFAULT NUL olors` varchar(32) DEFAULT NULL ` varchar(100) DEFAULT NULL , int(11) DEFAULT NULL , i` int(11) DEFAULT NULL, _id` int(11) DEFAULT NULL, i` int(11) DEFAULT NULL , nt` varchar(200) DEFAULT NULL, ` int(11) DEFAULT NULL , int(11) DEFAULT NULL , tation` int(11) DEFAULT NULL , rds_audit` varchar(3000) DEFAUL e_type` int(11) DEFAULT NULL, text, an_state` int(11) DEFAULT NULL, ed_time` timestamp NULL DEFAULT ed_by` varchar(36) DEFAULT NULL ed_time` timestamp NULL DEFAULT ed_by` varchar(36) DEFAULT NULL
