Thursday, March 25, 2010

Create User in MySQL

mysql> create user 'username' identified by 'password';

mysql> grant all privileges on 'dbname' to 'username'@localhost identified by 'password' with grant option;

mysql> grant select, insert, update, delete, create, drop
-> on db.'dbname'
-> to 'username'@localhost
-> identified by 'password';

Wednesday, March 24, 2010

[Wamp] Fatal error: Maximum execution time of 300 seconds exceeded

This is caused by the default configuration of wamp server. To modify this, you need to find libraries/config.default.php. You can see all default configurations here. Find the one you need to change; in this case, it should be:

$cfg['ExecTimeLimit'] integer [number of seconds]

Function of the value of this variable:
Set the number of seconds a script is allowed to run. If seconds is set to zero, no time limit is imposed.This setting is used while importing/exporting dump files but has no effect when PHP is running in safe mode.

Increase Limit on Dumpfile Size in MySQL

If you are using wamp server, please do the following:

> sudo gedit /etc/php5/apache2/php.ini

Modify
a) post_max_size
b) memory_limit
c) upload_max_filesize

In an order memory_limit > post_max_size (size greater than required DB) > upload_max_filesize Restart apache.

If you are using MySQL alone, please modify your "my.ini" file. If you cannot find such a file, please make a copy of your "my-medium.cnf" file and rename it to "my.ini". You'll find "max_allowed_packet" variable. Then change its value to a larger one you need!!!

Thursday, March 18, 2010

Hadoop中Writable接口的序列化

一种常用的序列化方法如下:
public static class Key implements WritableComparable { public String id = ""; public short weight; public Key() { } public Key(String i, short j) { id = i; weight = j; } @Override public void readFields(DataInput in) throws IOException { //先写字符串的长度信息 int length = in.readInt(); byte[] buf = new byte[length]; in.readFully(buf, 0, length); //得到id号 id = new String(buf); //得到权值 weight = in.readShort(); } @Override public void write(DataOutput out) throws IOException { String str = id.toString(); int length = str.length(); byte[] buf = str.getBytes(); //先写字符串长度 //WritableUtils.writeVInt(out, length); out.writeInt(length); //再写字符串数据 out.write(buf, 0, length); //接着是权值 out.writeShort(weight); } @Override public int hashCode() { return id.hashCode(); } @Override public String toString() { return id; } @Override public boolean equals(Object right) { //只要id相等就认为两个key相等 if (right instanceof Key) { Key r = (Key) right; return r.id.equals(id); } else { return false; } } @Override public int compareTo(Key k) { System.out.println("in compareTo, key=" + k.toString()); //先比较value id int cmp = id.compareTo(k.id); if (cmp != 0) { return cmp; } //如果value id相等,再比较权值 if (weight > k.weight) return -1; else if (weight < k.weight) return 1; else return 0; } }

解决Hadoop中的“输出目录已存在”的问题

Hadoop中使用Map Reduce时,每次运行的输出目录必须事先未建好,导致如果想使用相同的output目录,每次运行程序之前都要先删掉之前的输出目录。其实这一操作可以嵌到代码中执行:

FileSystem fstm = FileSystem.get(conf);
Path outDir = new Path(args[2]);
fstm.delete(outDir, true);


这样即可解决问题。

Wednesday, March 17, 2010

Linux下设置环境变量

JAVA:

$ export JAVA_HOME=/usr/java/jdk1.6.0_07
$ export PATH=$PATH:$JAVA_HOME/bin

hadoop:

First setup the HADOOP_HOME environment variable to the install directory. Then append $HADOOP_HOME/bin to your PATH environment variable. It is advisable to add these to profile setup scripts.
$ export HADOOP_HOME=/u/kduan/Desktop/hadoop-0.20.2
$ export PATH=$PATH:$HADOOP_HOME/bin

要注意的是,等号的左右不能出现空格。最好修改一下.bash_profile文件,把上述export命令添加进去,这样每次用户登录时自动执行export命令。