Skip to content

Not able to run in Kerberos safe environment. #23

@snchtgrvr

Description

@snchtgrvr

Hi, I am trying to run the spyglass api in a kerberos environment, i used the following code :
// read the input file
val pipe = TextLine( args("input") )
// split each line of the input file using any whitespace "\s+" as the separator
.flatMap('line -> 'WORD) { line : String =>
line.toLowerCase.split("\s+")
}
// get count of each word in the input file
.groupBy('WORD) { _.size }
// convert to uppercase for use with HBase Phoenix
.rename('size -> 'SIZE)
// create a new pipe with byte values rather than string/number values
val pipe2 = new HBasePipeWrapper(pipe).toBytesWritable(new Fields("WORD"), new Fields("SIZE"))
// write to the TEST table with WORD as the rowkey, CF as the columnFamily and SIZE as the only value column
.write(new HBaseSource("TEST", "myserver.mydomain.com:2181", 'WORD, List("CF"), List(new Fields("SIZE"))) )
}

Changing the "myserver.mydomain.com:2181" with my specific quorum values. this specific code launches and shows 2 mappers and 60 reducers are required. the the mappers goes well but the reducers get stucked at 57 and then keeps failing without killing the task.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions