Hadoop从入门到精通27:MapReduce高级功能——序列化

1.Java的序列化

如果一个类实现了Java的序列化接口(Serializable,标志接口,没有方法),这个类的对象可以作为输入和输出对象。

示例:将Student类实现序列化和反序列化。

//Student类文件:Student.java
package serializable.java;
import java.io.Serializable;
public class Student implements Serializable {
  /**
  * Student类实现了序列化
  */
  private static final long serialVersionUID = 1L;
  @Override
  public String toString() {
    return "Student [stuID=" + stuID + ", stuName=" + stuName + "]";
  }
  private int stuID;
  private String stuName;
  public Student() {
    super();
  }
  public int getStuID() {
    return stuID;
  }
  public void setStuID(int stuID) {
    this.stuID = stuID;
  }
  public String getStuName() {
    return stuName;
  }
  public void setStuName(String stuName) {
    this.stuName = stuName;
  }
}

//测试文件:TestMain.java
package serializable.java;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
public class TestMain {
  public static void main(String[] args) throws Exception {
    //序列化
    Student stu = new Student();
    stu.setStuID(1);
    stu.setStuName("Tom");
    OutputStream fileOut = new FileOutputStream("E:\\student.bin");
    ObjectOutputStream out = new ObjectOutputStream(fileOut);
    out.writeObject(stu);
    out.close();
    fileOut.close();

    //反序列化
    Student stu2 = new Student();
    InputStream fileIn = new FileInputStream("E:\\student.bin");
    ObjectInputStream in = new ObjectInputStream(fileIn);
    stu2 = (Student) in.readObject();
    System.out.println(stu2);
    in.close();
    fileIn.close();
  }
}

2.MR的序列化

如果一个类实现了MR的序列化的接口(Writable),这个类的对象可以作为Map和Reduce的输入和输出。

(1)示例:MR程序从HDFS读入员工信息,存入Employee类对象,并按指定格式输出到HDFS。

//Employee类:Employee.java
package serializable.mr;
import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;
import org.apache.hadoop.io.Writable;
public class Employee implements Writable {
  private int empno;
  private String ename;
  private String job;
  private int mgr;
  private String hiredate;
  private int sal;
  private int comm;
  private int deptno;
  @Override
  public void readFields(DataInput input) throws IOException {
    // 反序列化
    this.empno = input.readInt();
    this.ename = input.readUTF();
    this.job = input.readUTF();
    this.mgr = input.readInt();
    this.hiredate = input.readUTF();
    this.sal = input.readInt();
    this.comm = input.readInt();
    this.deptno = input.readInt();
  }
  @Override
  public void write(DataOutput output) throws IOException {
    // 序列化
    output.writeInt(this.empno);
    output.writeUTF(this.ename);
    output.writeUTF(this.job);
    output.writeInt(this.mgr);
    output.writeUTF(this.hiredate);
    output.writeInt(this.sal);
    output.writeInt(this.comm);
    output.writeInt(this.deptno);
  }
  public int getEmpno() {
    return empno;
  }
  public void setEmpno(int empno) {
    this.empno = empno;
  }
  public String getEname() {
    return ename;
  }
  public void setEname(String ename) {
    this.ename = ename;
  }
  public String getJob() {
    return job;
  }
  public void setJob(String job) {
    this.job = job;
  }
  public int getMgr() {
    return mgr;
  }
  public void setMgr(int mgr) {
    this.mgr = mgr;
  }
  public String getHiredate() {
    return hiredate;
  }
  public void setHiredate(String hiredate) {
    this.hiredate = hiredate;
  }
  public int getSal() {
    return sal;
  }
  public void setSal(int sal) {
    this.sal = sal;
  }
  public int getComm() {
    return comm;
  }
  @Override
  public String toString() {
    return "Employee [empno=" + empno + ", ename=" + ename + ", job=" + job + ", mgr=" + mgr + ", hiredate="
    + hiredate + ", sal=" + sal + ", comm=" + comm + ", deptno=" + deptno + "]";
  }
  public void setComm(int comm) {
    this.comm = comm;
  }
  public int getDeptno() {
    return deptno;
  }
  public void setDeptno(int deptno) {
    this.deptno = deptno;
  }
}
QR Code
微信扫一扫,欢迎咨询~

联系我们
武汉格发信息技术有限公司
湖北省武汉市经开区科技园西路6号103孵化器
电话:155-2731-8020 座机:027-59821821
邮件:tanzw@gofarlic.com
Copyright © 2023 Gofarsoft Co.,Ltd. 保留所有权利
遇到许可问题?该如何解决!?
评估许可证实际采购量? 
不清楚软件许可证使用数据? 
收到软件厂商律师函!?  
想要少购买点许可证,节省费用? 
收到软件厂商侵权通告!?  
有正版license,但许可证不够用,需要新购? 
联系方式 155-2731-8020
预留信息,一起解决您的问题
* 姓名:
* 手机:

* 公司名称:

姓名不为空

手机不正确

公司不为空