枫芸志

【PHP】统计中英文单词数（GB2312/UTF-8编码)

英文单词的统计可以直接用php原生的函数str_word_count来进行统计。但这个函数对于中文汉字显得无能为力，无法准确统计到汉字个数。

解决办法是根据汉字的编码规则，自己来实现中文汉字数统计和中英文单词数统计。汉字编码参考Unicode编码表和GB2312区位码、编码表与编码规则。

对于GB2312编码的字符采用以下函数：





<?php
define( "GB2312_CHINESE_PATTERN", "/[\xb0-\xfe][\xa0-\xfe]/" );
define( "GB2312_SYMBOL_PATTERN", "/[\xa1-\xa3][\xa0-\xfe]/" );
// count only chinese words
function str_gb2312_chinese_word_count($str = ""){
    $str = preg_replace(GB2312_SYMBOL_PATTERN, "", $str);
    return preg_match_all(GB2312_CHINESE_PATTERN, $str, $arr);
}
// count both chinese and english
function str_gb2312_mix_word_count($str = ""){
    $str = preg_replace(GB2312_SYMBOL_PATTERN, "", $str);
    return str_gb2312_chinese_word_count($str) + str_word_count(preg_replace(GB2312_CHINESE_PATTERN, "", $str));
}
?>

对于UTF-8编码的字符采用以下函数：

<?php
define( "UTF8_CHINESE_PATTERN", "/[\x{4e00}-\x{9fff}\x{f900}-\x{faff}]/u" );
define( "UTF8_SYMBOL_PATTERN", "/[\x{ff00}-\x{ffef}\x{2000}-\x{206F}]/u" );
// count only chinese words
function str_utf8_chinese_word_count($str = ""){
    $str = preg_replace(UTF8_SYMBOL_PATTERN, "", $str);
    return preg_match_all(UTF8_CHINESE_PATTERN, $str, $arr);
}
// count both chinese and english
function str_utf8_mix_word_count($str = ""){
    $str = preg_replace(UTF8_SYMBOL_PATTERN, "", $str);
    return str_utf8_chinese_word_count($str) + str_word_count(preg_replace(UTF8_CHINESE_PATTERN, "", $str));
}?>

以上两种代码功能相同，只是根据不同的字符编码做了不同的实现，实际使用视页面编码对应选择。都有两个函数，一个只统计中文汉字数，另一个统计中英文单词数（中文汉字数+英文单词数），中英文符号都不计入数字统计。

特别说明：如不先去除中文标点会导致统计出错，如GB2312编码下"：‘"两个中文标点的字节表示为a3baa1ae，中间部分baa1正好对应GB2312编码地"骸"字，会被统计为一个中文汉字，导致计数错误。

函数使用可参考以下测试页面：

<?php

define( "GB2312_CHINESE_PATTERN", "/[\xb0-\xfe][\xa0-\xfe]/" );
define( "GB2312_SYMBOL_PATTERN", "/[\xa1-\xa3][\xa0-\xfe]/" );
// count only chinese words
function str_gb2312_chinese_word_count($str = ""){
    $str = preg_replace(GB2312_SYMBOL_PATTERN, "", $str);
    return preg_match_all(GB2312_CHINESE_PATTERN, $str, $textrr);
}
// count both chinese and english
function str_gb2312_mix_word_count($str = ""){
    $str = preg_replace(GB2312_SYMBOL_PATTERN, "", $str);
    return str_gb2312_chinese_word_count($str) + str_word_count(preg_replace(GB2312_CHINESE_PATTERN, "", $str));
}

define( "UTF8_CHINESE_PATTERN", "/[\x{4e00}-\x{9fff}\x{f900}-\x{faff}]/u" );
define( "UTF8_SYMBOL_PATTERN", "/[\x{ff00}-\x{ffef}\x{2000}-\x{206F}]/u" );
// count only chinese words
function str_utf8_chinese_word_count($str = ""){
    $str = preg_replace(UTF8_SYMBOL_PATTERN, "", $str);
    return preg_match_all(UTF8_CHINESE_PATTERN, $str, $textrr);
}
// count both chinese and english
function str_utf8_mix_word_count($str = ""){
    $str = preg_replace(UTF8_SYMBOL_PATTERN, "", $str);
    return str_utf8_chinese_word_count($str) + str_word_count(preg_replace(UTF8_CHINESE_PATTERN, "", $str));
}

// convert a string to hex-coding form
function binhex($str) {
    $hex = "";
    $i = 0;
    do {
        $hex .= sprintf("%02x", ord($str{$i}));
        $i++;
    } while ($i < strlen($str));
    return $hex;
}

$text = $_REQUEST["text"] ? $_REQUEST["text"] : "";
echo "Text: " . $text . "<br />";
echo "Hex : " . ($text ? binhex($text) : "") . "<br />";
// use one of the following two lines according to the page encoding
echo "Word count: " . str_gb2312_mix_word_count($text);
// echo "Word count: " . str_utf8_mix_word_count($text);
?>

<form action="test.php">
<input type="text" name="text" id="text" value="<?=$text?>"/>
<input type="submit" />
</form>




本博客所有文章如无特别注明均为原创。
复制或转载请以超链接形式注明转自枫芸志，原文地址《【PHP】统计中英文单词数（GB2312/UTF-8编码)》
标签： PHP • 字符编码
分享：