{"id":167907,"date":"2022-04-10T01:22:57","date_gmt":"2022-04-09T17:22:57","guid":{"rendered":"http:\/\/www.idc.net\/help\/167907\/"},"modified":"2022-04-10T01:22:57","modified_gmt":"2022-04-09T17:22:57","slug":"%e5%90%8e%e6%b5%aa%e4%ba%91python%e6%95%99%e7%a8%8b%ef%bc%9aunicode-%e6%8c%87%e5%8d%97","status":"publish","type":"post","link":"https:\/\/idc.net\/help\/167907\/","title":{"rendered":"\u540e\u6d6a\u4e91Python\u6559\u7a0b\uff1aUnicode \u6307\u5357"},"content":{"rendered":"

Unicode \u6307\u5357<\/h1>\n

\u53d1\u5e03\u7248\u672c<\/p>\n

1.12<\/p>\n

\u672c\u6587\u4ecb\u7ecd\u4e86 Python \u5bf9\u8868\u793a\u6587\u672c\u6570\u636e\u7684 Unicode \u89c4\u8303\u7684\u652f\u6301\uff0c\u5e76\u5bf9\u5404\u79cd Unicode \u5e38\u89c1\u4f7f\u7528\u95ee\u9898\u505a\u4e86\u89e3\u91ca\u3002<\/p>\n

Unicode \u6982\u8ff0<\/h2>\n

\u5b9a\u4e49<\/h3>\n

\u5982\u4eca\u7684\u7a0b\u5e8f\u9700\u8981\u80fd\u591f\u5904\u7406\u5404\u79cd\u5404\u6837\u7684\u5b57\u7b26\u3002\u5e94\u7528\u7a0b\u5e8f\u901a\u5e38\u505a\u4e86\u56fd\u9645\u5316\u5904\u7406\uff0c\u7528\u6237\u53ef\u4ee5\u9009\u62e9\u4e0d\u540c\u7684\u8bed\u8a00\u663e\u793a\u4fe1\u606f\u548c\u8f93\u51fa\u6570\u636e\u3002\u540c\u4e00\u4e2a\u7a0b\u5e8f\u53ef\u80fd\u9700\u8981\u4ee5\u82f1\u8bed\u3001\u6cd5\u8bed\u3001\u65e5\u8bed\u3001\u5e0c\u4f2f\u6765\u8bed\u6216\u4fc4\u8bed\u8f93\u51fa\u9519\u8bef\u4fe1\u606f\u3002\u7f51\u9875\u5185\u5bb9\u53ef\u80fd\u7531\u8fd9\u4e9b\u8bed\u8a00\u4e66\u5199\uff0c\u5e76\u4e14\u53ef\u80fd\u5305\u542b\u4e0d\u540c\u7684\u8868\u60c5\u7b26\u53f7\u3002Python \u7684\u5b57\u7b26\u4e32\u7c7b\u578b\u91c7\u7528 Unicode \u6807\u51c6\u6765\u8868\u793a\u5b57\u7b26\uff0c\u4f7f\u5f97 Python \u7a0b\u5e8f\u80fd\u591f\u6b63\u5e38\u5904\u7406\u6240\u6709\u8fd9\u4e9b\u4e0d\u540c\u7684\u5b57\u7b26\u3002<\/p>\n

Unicode \u89c4\u8303\uff08https:\/\/www.unicode.org\/\uff09\u65e8\u5728\u7f57\u5217\u4eba\u7c7b\u8bed\u8a00\u6240\u7528\u5230\u7684\u6240\u6709\u5b57\u7b26\uff0c\u5e76\u8d4b\u4e88\u6bcf\u4e2a\u5b57\u7b26\u552f\u4e00\u7684\u7f16\u7801\u3002\u8be5\u89c4\u8303\u4e00\u76f4\u5728\u8fdb\u884c\u4fee\u8ba2\u548c\u66f4\u65b0\uff0c\u4e0d\u65ad\u52a0\u5165\u65b0\u7684\u8bed\u79cd\u548c\u7b26\u53f7\u3002<\/p>\n

\u4e00\u4e2a \u5b57\u7b26<\/strong> \u662f\u6587\u672c\u7684\u6700\u5c0f\u7ec4\u4ef6\u3002\u2018A\u2019\u3001\u2018B\u2019\u3001\u2018C\u2019 \u7b49\u90fd\u662f\u4e0d\u540c\u7684\u5b57\u7b26\u3002\u2018\u00c8\u2019 \u548c \u2018\u00cd\u2019 \u4e5f\u4e00\u6837\u3002\u5b57\u7b26\u4f1a\u968f\u7740\u8bed\u8a00\u6216\u8005\u4e0a\u4e0b\u6587\u7684\u53d8\u5316\u800c\u53d8\u5316\u3002\u6bd4\u5982\uff0c\u2018\u2160\u2019 \u662f\u4e00\u4e2a\u8868\u793a \u201c\u7f57\u9a6c\u6570\u5b57 1\u201d \u7684\u5b57\u7b26\uff0c\u5b83\u4e0e\u5927\u5199\u5b57\u6bcd \u2018I\u2019 \u4e0d\u540c\u3002\u4ed6\u4eec\u5f80\u5f80\u770b\u8d77\u6765\u76f8\u540c\uff0c\u4f46\u8fd9\u662f\u4e24\u4e2a\u6709\u7740\u4e0d\u540c\u542b\u4e49\u7684\u5b57\u7b26\u3002<\/p>\n

Unicode \u6807\u51c6\u63cf\u8ff0\u4e86\u5b57\u7b26\u662f\u5982\u4f55\u7528 \u7801\u4f4d\uff08code point\uff09<\/strong> \u8868\u793a\u7684\u3002\u7801\u4f4d\u7684\u53d6\u503c\u8303\u56f4\u662f 0 \u5230 0x10FFFF \u7684\u6574\u6570\uff08\u5927\u7ea6 110 \u4e07\u4e2a\u503c\uff0c\u5b9e\u9645\u5206\u914d\u7684\u6570\u5b57 \u6ca1\u6709\u90a3\u4e48\u591a\uff09\u3002\u5728 Unicode \u6807\u51c6\u548c\u672c\u6587\u4e2d\uff0c\u7801\u4f4d\u91c7\u7528 U+265E<\/code> \u7684\u5f62\u5f0f\uff0c\u8868\u793a\u503c\u4e3a 0x265e<\/code> \u7684\u5b57\u7b26\uff08\u5341\u8fdb\u5236\u4e3a 9822\uff09\u3002<\/p>\n

Unicode \u6807\u51c6\u4e2d\u5305\u542b\u4e86\u8bb8\u591a\u8868\u683c\uff0c\u5217\u51fa\u4e86\u5f88\u591a\u5b57\u7b26\u53ca\u5176\u5bf9\u5e94\u7684\u7801\u4f4d\u3002<\/p>\n

\n \n \n \n 
    \n  \n  \n  \n  
  1. 0061<\/span> <\/span>'a'<\/span>;<\/span> LATIN SMALL LETTER A<\/span><\/code><\/li>\n  \n  \n  \n  
  2. 0062<\/span> <\/span>'b'<\/span>;<\/span> LATIN SMALL LETTER B<\/span><\/code><\/li>\n  \n  \n  \n  
  3. 0063<\/span> <\/span>'c'<\/span>;<\/span> LATIN SMALL LETTER C<\/span><\/code><\/li>\n  \n  \n  \n  
  4. ...<\/span><\/code><\/li>\n  \n  \n  \n  
  5. 007B<\/span> <\/span>'{'<\/span>;<\/span> LEFT CURLY BRACKET<\/span><\/code><\/li>\n  \n  \n  \n  
  6. ...<\/span><\/code><\/li>\n  \n  \n  \n  
  7. 2167<\/span> <\/span>'\u2167'<\/span>;<\/span> ROMAN NUMERAL EIGHT<\/span><\/code><\/li>\n  \n  \n  \n  
  8. 2168<\/span> <\/span>'\u2168'<\/span>;<\/span> ROMAN NUMERAL NINE<\/span><\/code><\/li>\n  \n  \n  \n  
  9. ...<\/span><\/code><\/li>\n  \n  \n  \n  
  10. 265E<\/span> <\/span>''<\/span>;<\/span> BLACK CHESS KNIGHT<\/span><\/code><\/li>\n  \n  \n  \n  
  11. 265F<\/span> <\/span>''<\/span>;<\/span> BLACK CHESS PAWN<\/span><\/code><\/li>\n  \n  \n  \n  
  12. ...<\/span><\/code><\/li>\n  \n  \n  \n  
  13. 1F600<\/span> <\/span>''<\/span>;<\/span> GRINNING FACE<\/span><\/code><\/li>\n  \n  \n  \n  
  14. 1F609<\/span> <\/span>''<\/span>;<\/span> WINKING FACE<\/span><\/code><\/li>\n  \n  \n  \n  
  15. ...<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
    \u4e25\u683c\u5730\u8bf4\uff0c\u4e0a\u8ff0\u5b9a\u4e49\u6697\u793a\u4e86\u4ee5\u4e0b\u8bf4\u6cd5\u662f\u6ca1\u6709\u610f\u4e49\u7684\uff1a\u201c\u8fd9\u662f\u5b57\u7b26 U+265E<\/code>\u201d\u3002U+265E<\/code> \u53ea\u662f\u4e00\u4e2a\u7801\u4f4d\uff0c\u4ee3\u8868\u67d0\u4e2a\u7279\u5b9a\u7684\u5b57\u7b26\uff1b\u8fd9\u91cc\u5b83\u4ee3\u8868\u4e86\u5b57\u7b26 \u201c\u56fd\u9645\u8c61\u68cb\u9ed1\u9a91\u58eb\u201d \u2018\u2019\u3002\u5728\u975e\u6b63\u5f0f\u7684\u4e0a\u4e0b\u6587\u4e2d\uff0c\u6709\u65f6\u4f1a\u5ffd\u7565\u7801\u4f4d\u548c\u5b57\u7b26\u7684\u533a\u522b\u3002<\/p>\n
    \u4e00\u4e2a\u5b57\u7b26\u5728\u5c4f\u5e55\u6216\u7eb8\u4e0a\u88ab\u8868\u793a\u4e3a\u4e00\u7ec4\u56fe\u5f62\u5143\u7d20\uff0c\u88ab\u79f0\u4e3a \u5b57\u5f62\uff08glyph\uff09<\/strong> \u3002\u6bd4\u5982\uff0c\u5927\u5199\u5b57\u6bcd A \u7684\u5b57\u5f62\uff0c\u662f\u4e24\u7b14\u659c\u7ebf\u548c\u4e00\u7b14\u6a2a\u7ebf\uff0c\u800c\u5177\u4f53\u7684\u7ec6\u8282\u53d6\u51b3\u4e8e\u6240\u4f7f\u7528\u7684\u5b57\u4f53\u3002\u5927\u90e8\u5206 Python \u4ee3\u7801\u4e0d\u5fc5\u62c5\u5fc3\u5b57\u5f62\uff0c\u627e\u5230\u6b63\u786e\u7684\u663e\u793a\u5b57\u5f62\u901a\u5e38\u662f\u4ea4\u7ed9 GUI \u5de5\u5177\u5305\u6216\u7ec8\u7aef\u7684\u5b57\u4f53\u6e32\u67d3\u7a0b\u5e8f\u6765\u5b8c\u6210\u3002<\/p>\n
    \u7f16\u7801<\/h3>\n
    \u4e0a\u4e00\u6bb5\u53ef\u4ee5\u5f52\u7ed3\u4e3a\uff1a\u4e00\u4e2a Unicode \u5b57\u7b26\u4e32\u662f\u4e00\u7cfb\u5217\u7801\u4f4d\uff08\u4ece 0 \u5230 0x10FFFF<\/code> \u6216\u8005\u8bf4\u5341\u8fdb\u5236\u7684 1,114,111 \u7684\u6570\u5b57\uff09\u7ec4\u6210\u7684\u5e8f\u5217\u3002\u8fd9\u4e00\u5e8f\u5217\u5728\u5185\u5b58\u4e2d\u9700\u88ab\u8868\u793a\u4e3a\u4e00\u7ec4 \u7801\u5143\uff08code unit\uff09<\/strong> \uff0c \u7801\u5143<\/strong> \u4f1a\u6620\u5c04\u6210\u5305\u542b\u516b\u4e2a\u4e8c\u8fdb\u5236\u4f4d\u7684\u5b57\u8282\u3002\u5c06 Unicode \u5b57\u7b26\u4e32\u7ffb\u8bd1\u6210\u5b57\u8282\u5e8f\u5217\u7684\u89c4\u5219\u79f0\u4e3a \u5b57\u7b26\u7f16\u7801<\/strong> \uff0c\u6216\u8005 \u7f16\u7801<\/strong> \u3002<\/p>\n
    \u5927\u5bb6\u9996\u5148\u4f1a\u60f3\u5230\u7684\u7f16\u7801\u53ef\u80fd\u662f\u7528 32 \u4f4d\u7684\u6574\u6570\u4f5c\u4e3a\u4ee3\u7801\u4f4d\uff0c\u7136\u540e\u91c7\u7528 CPU \u5bf9 32 \u4f4d\u6574\u6570\u7684\u8868\u793a\u6cd5\u3002\u5b57\u7b26\u4e32 \u201cPython\u201d \u7528\u8fd9\u79cd\u8868\u793a\u6cd5\u53ef\u80fd\u4f1a\u5982\u4e0b\u6240\u793a\uff1a<\/p>\n
    \n \n \n \n 
      \n  \n  \n  \n  
    1.  P y t h o n<\/span><\/code><\/li>\n  \n  \n  \n  
    2. 0x50<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>79<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>74<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>68<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>6f<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>6e<\/span> <\/span>00<\/span> <\/span>00<\/span> <\/span>00<\/span><\/code><\/li>\n  \n  \n  \n  
    3.  <\/span>0<\/span> <\/span>1<\/span> <\/span>2<\/span> <\/span>3<\/span> <\/span>4<\/span> <\/span>5<\/span> <\/span>6<\/span> <\/span>7<\/span> <\/span>8<\/span> <\/span>9<\/span> <\/span>10<\/span> <\/span>11<\/span> <\/span>12<\/span> <\/span>13<\/span> <\/span>14<\/span> <\/span>15<\/span> <\/span>16<\/span> <\/span>17<\/span> <\/span>18<\/span> <\/span>19<\/span> <\/span>20<\/span> <\/span>21<\/span> <\/span>22<\/span> <\/span>23<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
      \u8fd9\u79cd\u8868\u793a\u6cd5\u975e\u5e38\u76f4\u767d\uff0c\u4f46\u4e5f\u5b58\u5728 \u4e00\u4e9b\u95ee\u9898\u3002<\/p>\n
        \n
      1. \n
        \u4e0d\u5177\u53ef\u79fb\u690d\u6027\uff1b\u4e0d\u540c\u7684\u5904\u7406\u5668\u7684\u5b57\u8282\u5e8f\u4e0d\u540c\u3002<\/p>\n<\/li>\n
      2. \n
        \u975e\u5e38\u6d6a\u8d39\u7a7a\u95f4\u3002 \u5728\u5927\u591a\u6570\u6587\u672c\u4e2d\uff0c\u5927\u90e8\u5206\u7801\u4f4d\u90fd\u5c0f\u4e8e 127 \u6216 255\uff0c\u56e0\u6b64\u5b57\u8282 0x00<\/code> \u5360\u7528\u4e86\u5927\u91cf\u7a7a\u95f4\u3002\u76f8\u8f83\u4e8e ASCII \u8868\u793a\u6cd5\u6240\u9700\u7684 6 \u4e2a\u5b57\u8282\uff0c\u4ee5\u4e0a\u5b57\u7b26\u4e32\u9700\u8981\u5360\u7528 24 \u4e2a\u5b57\u8282\u3002RAM \u7528\u91cf\u7684\u589e\u52a0\u6ca1\u90a3\u4e48\u8981\u7d27\uff08\u53f0\u5f0f\u8ba1\u7b97\u673a\u6709\u6210 GB \u7684 RAM\uff0c\u800c\u5b57\u7b26\u4e32\u901a\u5e38\u4e0d\u4f1a\u6709\u90a3\u4e48\u5927\uff09\uff0c\u4f46\u8981\u628a\u78c1\u76d8\u548c\u7f51\u7edc\u5e26\u5bbd\u7684\u7528\u91cf\u589e\u52a0 4 \u500d\u662f\u65e0\u6cd5\u5fcd\u53d7\u7684\u3002<\/p>\n<\/li>\n
      3. \n
        \u4e0e\u73b0\u6709\u7684 C \u51fd\u6570\uff08\u5982 strlen()<\/code> \uff09\u4e0d\u517c\u5bb9\uff0c\u56e0\u6b64\u9700\u8981\u91c7\u7528\u4e00\u5957\u65b0\u7684\u5bbd\u5b57\u7b26\u4e32\u51fd\u6570\u3002<\/p>\n<\/li>\n<\/ol>\n
        \u56e0\u6b64\u8fd9\u79cd\u7f16\u7801\u7528\u5f97\u4e0d\u591a\uff0c\u4eba\u4eec\u8f6c\u800c\u9009\u62e9\u5176\u4ed6\u66f4\u9ad8\u6548\u3001\u66f4\u65b9\u4fbf\u7684\u7f16\u7801\uff0c\u6bd4\u5982 UTF-8\u3002<\/p>\n
        UTF-8 \u662f\u6700\u5e38\u7528\u7684\u7f16\u7801\u4e4b\u4e00\uff0cPython \u5f80\u5f80\u9ed8\u8ba4\u4f1a\u91c7\u7528\u5b83\u3002UTF \u4ee3\u8868\u201cUnicode Transformation Format\u201d\uff0c\u20198\u2019 \u8868\u793a\u7f16\u7801\u91c7\u7528 8 \u4f4d\u6570\u3002\uff08UTF-16 \u548c UTF-32 \u7f16\u7801\u4e5f\u662f\u5b58\u5728\u7684\uff0c\u4f46\u5176\u4f7f\u7528\u9891\u7387\u4e0d\u5982 UTF-8\u3002\uff09UTF-8 \u7684\u89c4\u5219\u5982\u4e0b\uff1a<\/p>\n
          \n
        1. \n
          \u5982\u679c\u7801\u4f4d < 128\uff0c\u5219\u76f4\u63a5\u7528\u5bf9\u5e94\u7684\u5b57\u8282\u503c\u8868\u793a\u3002<\/p>\n<\/li>\n
        2. \n
          \u5982\u679c\u7801\u4f4d >= 128\uff0c\u5219\u8f6c\u6362\u4e3a 2\u30013\u30014 \u4e2a\u5b57\u8282\u7684\u5e8f\u5217\uff0c\u6bcf\u4e2a\u5b57\u8282\u503c\u90fd\u4f4d\u4e8e 128 \u548c 255 \u4e4b\u95f4\u3002<\/p>\n<\/li>\n<\/ol>\n
          UTF-8 \u6709\u51e0\u4e2a\u5f88\u65b9\u4fbf\u7684\u7279\u6027\uff1a<\/p>\n
            \n
          1. \n
            \u53ef\u4ee5\u5904\u7406\u4efb\u4f55 Unicode \u7801\u4f4d\u3002<\/p>\n<\/li>\n
          2. \n
            Unicode \u5b57\u7b26\u4e32\u88ab\u8f6c\u6362\u4e3a\u4e00\u4e2a\u5b57\u8282\u5e8f\u5217\uff0c\u4ec5\u5728\u8868\u793a\u7a7a\uff08null \uff09\u5b57\u7b26\uff08U+0000\uff09\u65f6\u624d\u4f1a\u5305\u542b\u96f6\u503c\u7684\u5b57\u8282\u3002\u8fd9\u610f\u5473\u7740 strcpy()<\/code> \u4e4b\u7c7b\u7684C \u51fd\u6570\u53ef\u4ee5\u5904\u7406 UTF-8 \u5b57\u7b26\u4e32\uff0c\u800c\u4e14\u7528\u90a3\u4e9b\u4e0d\u80fd\u5904\u7406\u5b57\u7b26\u4e32\u7ed3\u675f\u7b26\u4e4b\u5916\u7684\u96f6\u503c\u5b57\u8282\u7684\u534f\u8bae\u4e5f\u80fd\u53d1\u9001\u3002<\/p>\n<\/li>\n
          3. \n
            ASCII \u5b57\u7b26\u4e32\u4e5f\u662f\u4e5f\u662f\u4e5f\u662f\u5408\u6cd5\u7684 UTF-8 \u6587\u672c\u3002<\/p>\n<\/li>\n
          4. \n
            UTF-8 \u76f8\u5f53\u7d27\u51d1\uff1b\u5927\u591a\u6570\u5e38\u7528\u5b57\u7b26\u5747\u53ef\u7528\u4e00\u4e24\u4e2a\u5b57\u8282\u8868\u793a\u3002<\/p>\n<\/li>\n
          5. \n
            \u5982\u679c\u5b57\u8282\u6570\u636e\u88ab\u635f\u574f\u6216\u4e22\u5931\uff0c\u5219\u53ef\u4ee5\u627e\u51fa\u4e0b\u4e00\u4e2a UTF-8 \u7801\u70b9\u7684\u5f00\u59cb\u4f4d\u7f6e\u5e76\u91cd\u65b0\u5f00\u59cb\u540c\u6b65\u3002\u968f\u673a\u7684 8 \u4f4d\u6570\u636e\u4e5f\u4e0d\u592a\u53ef\u80fd\u50cf\u662f\u6709\u6548\u7684 UTF-8 \u7f16\u7801\u3002<\/p>\n<\/li>\n
          6. \n
            UTF-8 \u662f\u4e00\u79cd\u9762\u5411\u5b57\u8282\u7684\u7f16\u7801\u3002\u7f16\u7801\u89c4\u5b9a\u4e86\u6bcf\u4e2a\u5b57\u7b26\u7531\u4e00\u4e2a\u6216\u591a\u4e2a\u5b57\u8282\u7684\u5e8f\u5217\u8868\u793a\u3002\u8fd9\u907f\u514d\u4e86\u6574\u6570\u548c\u53cc\u5b57\u8282\u7f16\u7801\uff08\u5982 UTF-16 \u548c UTF-32\uff09\u53ef\u80fd\u51fa\u73b0\u7684\u5b57\u8282\u987a\u5e8f\u95ee\u9898\uff0c\u90a3\u65f6\u7684\u5b57\u8282\u5e8f\u5217\u4f1a\u56e0\u6267\u884c\u7f16\u7801\u7684\u786c\u4ef6\u800c\u5f02\u3002<\/p>\n<\/li>\n<\/ol>\n
            \u53c2\u8003\u6587\u732e<\/h3>\n
            Unicode Consortium \u7ad9\u70b9 \u5305\u542b Unicode \u89c4\u8303\u7684\u5b57\u7b26\u56fe\u8868\u3001\u8bcd\u6c47\u8868\u548c PDF \u7248\u672c\u3002\u8bf7\u505a\u597d\u51c6\u5907\uff0c\u6709\u4e9b\u5185\u5bb9\u8bfb\u8d77\u6765\u6709\u70b9\u96be\u5ea6\u3002\u8be5\u7f51\u7ad9\u4e0a\u8fd8\u63d0\u4f9b\u4e86 Unicode \u8d77\u6e90\u548c\u53d1\u5c55\u7684`\u5e74\u8868 <https:\/\/www.unicode.org\/history\/>`_ \u3002<\/p>\n
            \u5728 Computerphile \u7684 Youtube \u9891\u9053\u4e0a\uff0cTom Scott \u7b80\u8981\u5730`\u8ba8\u8bba\u4e86 Unicode \u548c UTF-8 <https:\/\/www.youtube.com\/watch?v=MijmeoH9LT4>`_\uff089 \u5206 36 \u79d2\uff09\u7684\u5386\u53f2\u3002<\/p>\n
            To help understand the standard, Jukka Korpela has written an introductory guide to reading the Unicode character tables.<\/p>\n
            Joel Spolsky \u64b0\u5199\u4e86\u53e6\u4e00\u7bc7\u4e0d\u9519\u7684\u4ecb\u7ecd\u6027\u6587\u7ae0 <https:\/\/www.joelonsoftware.com\/2003\/10\/08\/the-absolute-minimum-every-software-developer-absolutely-positively-must-know-about-unicode-and-character- set-no-excuses\/>`_ \u3002\u5982\u679c\u672c\u6587\u6ca1\u8ba9\u60a8\u5f04\u6e05\u695a\uff0c\u90a3\u5e94\u5728\u7ee7\u7eed\u4e4b\u524d\u5148\u8bd5\u7740\u8bfb\u8bfb\u8fd9\u7bc7\u6587\u7ae0\u3002<\/p>\n
            Wikipedia \u6761\u76ee\u901a\u5e38\u4e5f\u6709\u5e2e\u52a9\uff1b\u8bf7\u53c2\u9605\u201c\u5b57\u7b26\u7f16\u7801\u201d\u548c UTF-8 \u7684\u6761\u76ee\uff0c\u4f8b\u5982\uff1a<\/p>\n
            Python\u5bf9Unicode\u7684\u652f\u6301<\/h2>\n
            \u73b0\u5728\u60a8\u5df2\u7ecf\u4e86\u89e3\u4e86 Unicode \u7684\u57fa\u7840\u77e5\u8bc6\uff0c\u53ef\u4ee5\u770b\u4e0b Python \u7684 Unicode \u7279\u6027\u3002<\/p>\n
            \u5b57\u7b26\u4e32\u7c7b\u578b<\/h3>\n
            \u4ece Python 3.0 \u5f00\u59cb\uff0c str \u7c7b\u578b\u5305\u542b\u4e86 Unicode \u5b57\u7b26\uff0c\u8fd9\u610f\u5473\u7740\u7528``\u201cunicode rocks!\u201d\u3001<\/code>\u2018unicode rocks!\u2019`` \u6216\u4e09\u91cd\u5f15\u53f7\u5b57\u7b26\u4e32\u8bed\u6cd5\u521b\u5efa\u7684\u4efb\u4f55\u5b57\u7b26\u4e32\u90fd\u4f1a\u5b58\u50a8\u4e3a Unicode\u3002<\/p>\n
            Python \u6e90\u4ee3\u7801\u7684\u9ed8\u8ba4\u7f16\u7801\u662f UTF-8\uff0c\u56e0\u6b64\u53ef\u4ee5\u76f4\u63a5\u5728\u5b57\u7b26\u4e32\u4e2d\u5305\u542b Unicode \u5b57\u7b26\uff1a<\/p>\n
            \n \n \n \n 
              \n  \n  \n  \n  
            1. try<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
            2.  <\/span>with<\/span> open<\/span>(<\/span>'\/tmp\/input.txt'<\/span>,<\/span> <\/span>'r'<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
            3.  <\/span>...<\/span><\/code><\/li>\n  \n  \n  \n  
            4. except<\/span> <\/span>OSError<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
            5.  <\/span># 'File not found' error message.<\/span><\/code><\/li>\n  \n  \n  \n  
            6.  <\/span>print<\/span>(<\/span>\"Fichier non trouv\u00e9\"<\/span>)<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
              \u65c1\u6ce8\uff1aPython 3 \u8fd8\u652f\u6301\u5728\u6807\u8bc6\u7b26\u4e2d\u4f7f\u7528 Unicode \u5b57\u7b26\uff1a<\/p>\n
              \n \n \n \n 
                \n  \n  \n  \n  
              1. r<\/span>\u00e9<\/span>pertoire <\/span>=<\/span> <\/span>\"\/tmp\/records.log\"<\/span><\/code><\/li>\n  \n  \n  \n  
              2. with<\/span> open<\/span>(<\/span>r<\/span>\u00e9<\/span>pertoire<\/span>,<\/span> <\/span>\"w\"<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
              3.  f<\/span>.<\/span>write<\/span>(<\/span>\"test\\n\"<\/span>)<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                \u5982\u679c\u65e0\u6cd5\u5728\u7f16\u8f91\u5668\u4e2d\u8f93\u5165\u67d0\u4e2a\u5b57\u7b26\uff0c\u6216\u51fa\u4e8e\u67d0\u79cd\u539f\u56e0\u60f3\u53ea\u4fdd\u7559 ASCII \u7f16\u7801\u7684\u6e90\u4ee3\u7801\uff0c\u5219\u8fd8\u53ef\u4ee5\u5728\u5b57\u7b26\u4e32\u4e2d\u4f7f\u7528\u8f6c\u4e49\u5e8f\u5217\u3002\uff08\u6839\u636e\u7cfb\u7edf\u7684\u4e0d\u540c\uff0c\u53ef\u80fd\u4f1a\u770b\u5230\u771f\u7684\u5927\u5199 Delta \u5b57\u4f53\u800c\u4e0d\u662f u \u8f6c\u4e49\u7b26\u3002\uff09\uff1a<\/p>\n
                \n \n \n \n 
                  \n  \n  \n  \n  
                1. >>><\/span> <\/span>\"\\N{GREEK CAPITAL LETTER DELTA}\"<\/span> <\/span># Using the character name<\/span><\/code><\/li>\n  \n  \n  \n  
                2. '\\u0394'<\/span><\/code><\/li>\n  \n  \n  \n  
                3. >>><\/span> <\/span>\"\\u0394\"<\/span> <\/span># Using a 16-bit hex value<\/span><\/code><\/li>\n  \n  \n  \n  
                4. '\\u0394'<\/span><\/code><\/li>\n  \n  \n  \n  
                5. >>><\/span> <\/span>\"\\U00000394\"<\/span> <\/span># Using a 32-bit hex value<\/span><\/code><\/li>\n  \n  \n  \n  
                6. '\\u0394'<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                  \u6b64\u5916\uff0c\u53ef\u4ee5\u7528 bytes \u7684 decode() \u65b9\u6cd5\u521b\u5efa\u4e00\u4e2a\u5b57\u7b26\u4e32\u3002 \u8be5\u65b9\u6cd5\u53ef\u4ee5\u63a5\u53d7 encoding<\/em> \u53c2\u6570\uff0c\u6bd4\u5982\u53ef\u4ee5\u4e3a UTF-8<\/code> \uff0c\u4ee5\u53ca\u53ef\u9009\u7684 errors<\/em> \u53c2\u6570\u3002<\/p>\n
                  \u82e5\u65e0\u6cd5\u6839\u636e\u7f16\u7801\u89c4\u5219\u5bf9\u8f93\u5165\u5b57\u7b26\u4e32\u8fdb\u884c\u7f16\u7801\uff0cerrors<\/em> \u53c2\u6570\u6307\u5b9a\u4e86\u54cd\u5e94\u7b56\u7565\u3002 \u8be5\u53c2\u6570\u7684\u5408\u6cd5\u503c\u53ef\u4ee5\u662f 'strict'<\/code> (\u89e6\u53d1 UnicodeDecodeError \u5f02\u5e38)\u3001'replace'<\/code> (\u7528 U+FFFD<\/code>\u3001REPLACEMENT CHARACTER<\/code>)\u3001'ignore'<\/code> (\u53ea\u662f\u5c06\u5b57\u7b26\u4ece Unicode \u7ed3\u679c\u4e2d\u53bb\u6389)\uff0c\u6216 'backslashreplace'<\/code> (\u63d2\u5165\u4e00\u4e2a \\xNN<\/code> \u8f6c\u4e49\u5e8f\u5217)\u3002 \u4ee5\u4e0b\u793a\u4f8b\u6f14\u793a\u4e86\u8fd9\u4e9b\u4e0d\u540c\u7684\u53c2\u6570:<\/p>\n
                  \n \n \n \n 
                    \n  \n  \n  \n  
                  1. >>><\/span> b<\/span>'\\x80abc'<\/span>.<\/span>decode<\/span>(<\/span>\"utf-8\"<\/span>,<\/span> <\/span>\"strict\"<\/span>)<\/span> <\/span><\/code><\/li>\n  \n  \n  \n  
                  2. Traceback<\/span> <\/span>(<\/span>most recent call <\/span>last<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                  3.  <\/span>...<\/span><\/code><\/li>\n  \n  \n  \n  
                  4. UnicodeDecodeError<\/span>:<\/span> <\/span>'utf-8'<\/span> codec can<\/span>'t decode byte 0x80 in position 0:<\/span><\/code><\/li>\n  \n  \n  \n  
                  5.  invalid start byte<\/span><\/code><\/li>\n  \n  \n  \n  
                  6. >>> b'<\/span>\\x<\/span>80abc<\/span>'.decode(\"utf-8\", \"replace\")<\/span><\/code><\/li>\n  \n  \n  \n  
                  7. '<\/span>\\ufffdabc<\/span>'<\/span><\/code><\/li>\n  \n  \n  \n  
                  8. >>> b'<\/span>\\x<\/span>80abc<\/span>'.decode(\"utf-8\", \"backslashreplace\")<\/span><\/code><\/li>\n  \n  \n  \n  
                  9. '<\/span>\\\\x80abc<\/span>'<\/span><\/code><\/li>\n  \n  \n  \n  
                  10. >>> b'<\/span>\\x<\/span>80abc<\/span>'.decode(\"utf-8\", \"ignore\")<\/span><\/code><\/li>\n  \n  \n  \n  
                  11. '<\/span>abc<\/span>'<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                    \u7f16\u7801\u683c\u5f0f\u4ee5\u5305\u542b\u7f16\u7801\u683c\u5f0f\u540d\u79f0\u7684\u5b57\u7b26\u4e32\u6765\u6307\u660e\u3002 Python \u6709\u5927\u7ea6 100 \u79cd\u4e0d\u540c\u7684\u7f16\u7801\u683c\u5f0f\uff1b\u6e05\u5355\u8be6\u89c1 Python \u5e93\u53c2\u8003\u6587\u6863 \u6807\u51c6\u7f16\u7801\u3002 \u4e00\u4e9b\u7f16\u7801\u683c\u5f0f\u6709\u591a\u4e2a\u540d\u79f0\uff0c\u6bd4\u5982 'latin-1'<\/code>\u3001'iso_8859_1'<\/code> \u548c '8859<\/code> \u90fd\u662f\u6307\u540c\u4e00\u79cd\u7f16\u7801\u3002<\/p>\n
                    \u5229\u7528\u5185\u7f6e\u51fd\u6570 chr() \u8fd8\u53ef\u4ee5\u521b\u5efa\u5355\u5b57\u7b26\u7684 Unicode \u5b57\u7b26\u4e32\uff0c\u8be5\u51fd\u6570\u53ef\u63a5\u53d7\u6574\u6570\u53c2\u6570\uff0c\u5e76\u8fd4\u56de\u5305\u542b\u5bf9\u5e94\u7801\u4f4d\u7684\u957f\u5ea6\u4e3a 1 \u7684 Unicode \u5b57\u7b26\u4e32\u3002\u5185\u7f6e\u51fd\u6570 ord() \u662f\u5176\u9006\u64cd\u4f5c\uff0c\u53c2\u6570\u4e3a\u5355\u4e2a\u5b57\u7b26\u7684 Unicode \u5b57\u7b26\u4e32\uff0c\u5e76\u8fd4\u56de\u7801\u4f4d\u503c\uff1a<\/p>\n
                    \n \n \n \n 
                      \n  \n  \n  \n  
                    1. >>><\/span> chr<\/span>(<\/span>57344<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                    2. '\\ue000'<\/span><\/code><\/li>\n  \n  \n  \n  
                    3. >>><\/span> ord<\/span>(<\/span>'\\ue000'<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                    4. 57344<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                      \u8f6c\u6362\u4e3a\u5b57\u8282<\/h3>\n
                      bytes.decode() \u7684\u9006\u65b9\u6cd5\u662f str.encode() \uff0c\u5b83\u4f1a\u8fd4\u56de Unicode \u5b57\u7b26\u4e32\u7684 bytes \u5f62\u5f0f\uff0c\u5df2\u6309\u8981\u6c42\u7684 encoding<\/em> \u8fdb\u884c\u4e86\u7f16\u7801\u3002<\/p>\n
                      \u53c2\u6570 errors<\/em> \u7684\u610f\u4e49\u4e0e decode() \u65b9\u6cd5\u76f8\u540c\uff0c\u4f46\u652f\u6301\u66f4\u591a\u53ef\u80fd\u7684handler\u3002\u9664\u4e86 'strict'<\/code> \u3001 'ignore'<\/code> \u548c 'replace'<\/code> \uff08\u8fd9\u65f6\u4f1a\u63d2\u5165\u95ee\u53f7\u66ff\u6362\u6389\u65e0\u6cd5\u7f16\u7801\u7684\u5b57\u7b26\uff09\uff0c\u8fd8\u6709 'xmlcharrefreplace'<\/code> \uff08\u63d2\u5165\u4e00\u4e2a XML \u5b57\u7b26\u5f15\u7528\uff09\u3001 backslashreplace<\/code> \uff08\u63d2\u5165\u4e00\u4e2a \\uNNNN<\/code> \u8f6c\u4e49\u5e8f\u5217\uff09\u548c namereplace<\/code> \uff08\u63d2\u5165\u4e00\u4e2a \\N{...}<\/code> \u8f6c\u4e49\u5e8f\u5217 \uff09\u3002<\/p>\n
                      \u4ee5\u4e0b\u4f8b\u5b50\u6f14\u793a\u4e86\u5404\u79cd\u4e0d\u540c\u7684\u7ed3\u679c\uff1a<\/p>\n
                      \n \n \n \n 
                        \n  \n  \n  \n  
                      1. >>><\/span> u <\/span>=<\/span> chr<\/span>(<\/span>40960<\/span>)<\/span> <\/span>+<\/span> <\/span>'abcd'<\/span> <\/span>+<\/span> chr<\/span>(<\/span>1972<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                      2. >>><\/span> u<\/span>.<\/span>encode<\/span>(<\/span>'utf-8'<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                      3. b<\/span>'\\xea\\x80\\x80abcd\\xde\\xb4'<\/span><\/code><\/li>\n  \n  \n  \n  
                      4. >>><\/span> u<\/span>.<\/span>encode<\/span>(<\/span>'ascii'<\/span>)<\/span> <\/span><\/code><\/li>\n  \n  \n  \n  
                      5. Traceback<\/span> <\/span>(<\/span>most recent call <\/span>last<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                      6.  <\/span>...<\/span><\/code><\/li>\n  \n  \n  \n  
                      7. UnicodeEncodeError<\/span>:<\/span> <\/span>'ascii'<\/span> codec can<\/span>'t encode character '<\/span>\\ua000<\/span>' in<\/span><\/code><\/li>\n  \n  \n  \n  
                      8.  position 0: ordinal not in range(128)<\/span><\/code><\/li>\n  \n  \n  \n  
                      9. >>> u.encode('<\/span>ascii<\/span>', '<\/span>ignore<\/span>')<\/span><\/code><\/li>\n  \n  \n  \n  
                      10. b'<\/span>abcd<\/span>'<\/span><\/code><\/li>\n  \n  \n  \n  
                      11. >>> u.encode('<\/span>ascii<\/span>', '<\/span>replace<\/span>')<\/span><\/code><\/li>\n  \n  \n  \n  
                      12. b'<\/span>?<\/span>abcd<\/span>?<\/span>'<\/span><\/code><\/li>\n  \n  \n  \n  
                      13. >>> u.encode('<\/span>ascii<\/span>', '<\/span>xmlcharrefreplace<\/span>')<\/span><\/code><\/li>\n  \n  \n  \n  
                      14. b'<\/span>&#<\/span>40960<\/span>;<\/span>abcd<\/span>&#<\/span>1972<\/span>;<\/span>'<\/span><\/code><\/li>\n  \n  \n  \n  
                      15. >>> u.encode('<\/span>ascii<\/span>', '<\/span>backslashreplace<\/span>')<\/span><\/code><\/li>\n  \n  \n  \n  
                      16. b'<\/span>\\\\ua000abcd\\\\u07b4<\/span>'<\/span><\/code><\/li>\n  \n  \n  \n  
                      17. >>> u.encode('<\/span>ascii<\/span>', '<\/span>namereplace<\/span>')<\/span><\/code><\/li>\n  \n  \n  \n  
                      18. b'<\/span>\\\\N<\/span>{<\/span>YI SYLLABLE IT<\/span>}<\/span>abcd\\\\u07b4<\/span>'<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                        \u7528\u4e8e\u6ce8\u518c\u548c\u8bbf\u95ee\u53ef\u7528\u7f16\u7801\u683c\u5f0f\u7684\u5e95\u5c42\u51fd\u6570\uff0c\u4f4d\u4e8e codecs \u6a21\u5757\u4e2d\u3002 \u82e5\u8981\u5b9e\u73b0\u65b0\u7684\u7f16\u7801\u683c\u5f0f\uff0c\u5219\u8fd8\u9700\u8981\u4e86\u89e3 codecs \u6a21\u5757\u3002 \u4e0d\u8fc7\u8be5\u6a21\u5757\u8fd4\u56de\u7684\u7f16\u7801\u548c\u89e3\u7801\u51fd\u6570\u901a\u5e38\u66f4\u4e3a\u5e95\u5c42\u4e00\u4e9b\uff0c\u4e0d\u5927\u597d\u7528\uff0c\u7f16\u5199\u65b0\u7684\u7f16\u7801\u683c\u5f0f\u662f\u4e00\u9879\u4e13\u4e1a\u7684\u4efb\u52a1\uff0c\u56e0\u6b64\u672c\u6587\u4e0d\u4f1a\u6d89\u53ca\u8be5\u6a21\u5757\u3002<\/p>\n
                        Python \u6e90\u4ee3\u7801\u4e2d\u7684 Unicode \u6587\u5b57<\/h3>\n
                        \u5728 Python \u6e90\u4ee3\u7801\u4e2d\uff0c\u53ef\u4ee5\u7528 \\u<\/code> \u8f6c\u4e49\u5e8f\u5217\u4e66\u5199\u7279\u5b9a\u7684 Unicode \u7801\u4f4d\uff0c\u8be5\u5e8f\u5217\u540e\u8ddf 4 \u4e2a\u4ee3\u8868\u7801\u4f4d\u7684\u5341\u516d\u8fdb\u5236\u6570\u5b57\u3002\\U<\/code> \u8f6c\u4e49\u5e8f\u5217\u7528\u6cd5\u7c7b\u4f3c\uff0c\u4f46\u8981\u75288 \u4e2a\u5341\u516d\u8fdb\u5236\u6570\u5b57\uff0c\u800c\u4e0d\u662f 4 \u4e2a\uff1a<\/p>\n
                        \n \n \n \n 
                          \n  \n  \n  \n  
                        1. >>><\/span> s <\/span>=<\/span> <\/span>\"a\\xac\\u1234\\u20ac\\U00008000\"<\/span><\/code><\/li>\n  \n  \n  \n  
                        2. ...<\/span> <\/span># ^^^^ two-digit hex escape<\/span><\/code><\/li>\n  \n  \n  \n  
                        3. ...<\/span> <\/span># ^^^^^^ four-digit Unicode escape<\/span><\/code><\/li>\n  \n  \n  \n  
                        4. ...<\/span> <\/span># ^^^^^^^^^^ eight-digit Unicode escape<\/span><\/code><\/li>\n  \n  \n  \n  
                        5. >>><\/span> <\/span>[<\/span>ord<\/span>(<\/span>c<\/span>)<\/span> <\/span>for<\/span> c <\/span>in<\/span> s<\/span>]<\/span><\/code><\/li>\n  \n  \n  \n  
                        6. [<\/span>97<\/span>,<\/span> <\/span>172<\/span>,<\/span> <\/span>4660<\/span>,<\/span> <\/span>8364<\/span>,<\/span> <\/span>32768<\/span>]<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                          \u5bf9\u5927\u4e8e 127 \u7684\u7801\u4f4d\u4f7f\u7528\u8f6c\u4e49\u5e8f\u5217\uff0c\u6570\u91cf\u4e0d\u591a\u65f6\u6ca1\u4ec0\u4e48\u95ee\u9898\uff0c\u4f46\u5982\u679c\u8981\u7528\u5230\u5f88\u591a\u91cd\u97f3\u5b57\u7b26\uff0c\u8fd9\u4f1a\u53d8\u5f97\u5f88\u70e6\u4eba\uff0c\u7c7b\u4f3c\u4e8e\u7a0b\u5e8f\u4e2d\u7684\u4fe1\u606f\u662f\u7528\u6cd5\u8bed\u6216\u5176\u4ed6\u4f7f\u7528\u91cd\u97f3\u7684\u8bed\u8a00\u5199\u7684\u3002\u4e5f\u53ef\u4ee5\u7528\u5185\u7f6e\u51fd\u6570 chr() \u62fc\u88c5\u5b57\u7b26\u4e32\uff0c\u4f46\u4f1a\u66f4\u52a0\u4e4f\u5473\u3002<\/p>\n
                          \u7406\u60f3\u60c5\u51b5\u4e0b\uff0c\u90fd\u5e0c\u671b\u80fd\u7528\u6bcd\u8bed\u7684\u7f16\u7801\u4e66\u5199\u6587\u672c\u3002\u8fd8\u80fd\u7528\u559c\u597d\u7684\u7f16\u8f91\u5668\u7f16\u8f91 Python \u6e90\u4ee3\u7801\uff0c\u7f16\u8f91\u5668\u8981\u80fd\u81ea\u7136\u5730\u663e\u793a\u91cd\u97f3\u7b26\uff0c\u5e76\u5728\u8fd0\u884c\u65f6\u4f7f\u7528\u6b63\u786e\u7684\u5b57\u7b26\u3002<\/p>\n
                          \u9ed8\u8ba4\u60c5\u51b5\u4e0b\uff0cPython \u652f\u6301\u4ee5 UTF-8 \u683c\u5f0f\u7f16\u5199\u6e90\u4ee3\u7801\uff0c\u4f46\u5982\u679c\u58f0\u660e\u8981\u7528\u7684\u7f16\u7801\uff0c\u5219\u51e0\u4e4e\u53ef\u4ee5\u4f7f\u7528\u4efb\u4f55\u7f16\u7801\u3002\u53ea\u8981\u5728\u6e90\u6587\u4ef6\u7684\u7b2c\u4e00\u884c\u6216\u7b2c\u4e8c\u884c\u5305\u542b\u4e00\u4e2a\u7279\u6b8a\u6ce8\u91ca\u5373\u53ef\uff1a<\/p>\n
                          \n \n \n \n 
                            \n  \n  \n  \n  
                          1. #!\/usr\/bin\/env python<\/span><\/code><\/li>\n  \n  \n  \n  
                          2. # -*- coding: latin-1 -*-<\/span><\/code><\/li>\n  \n  \n  \n  
                          3. <\/code><\/li>\n  \n  \n  \n  
                          4. u <\/span>=<\/span> <\/span>'abcd\u00e9'<\/span><\/code><\/li>\n  \n  \n  \n  
                          5. print<\/span>(<\/span>ord<\/span>(<\/span>u<\/span>[-<\/span>1<\/span>]))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                            \u4e0a\u8ff0\u8bed\u6cd5\u7684\u7075\u611f\u6765\u81ea\u4e8e Emacs \u7528\u4e8e\u6307\u5b9a\u6587\u4ef6\u5c40\u90e8\u53d8\u91cf\u7684\u7b26\u53f7\u3002Emacs \u652f\u6301\u8bb8\u591a\u4e0d\u540c\u7684\u53d8\u91cf\uff0c\u4f46 Python \u4ec5\u652f\u6301\u201c\u7f16\u7801\u201d\u3002 -*-<\/code> \u7b26\u53f7\u5411 Emacs \u6807\u660e\u8be5\u6ce8\u91ca\u662f\u7279\u6b8a\u7684\uff1b\u8fd9\u5bf9 Python \u6ca1\u6709\u4ec0\u4e48\u610f\u4e49\uff0c\u53ea\u662f\u4e00\u79cd\u7ea6\u5b9a\u3002Python \u4f1a\u5728\u6ce8\u91ca\u4e2d\u67e5\u627e coding: name<\/code> \u6216 coding=name<\/code> \u3002<\/p>\n
                            \u5982\u679c\u6ca1\u6709\u8fd9\u79cd\u6ce8\u91ca\uff0c\u5219\u9ed8\u8ba4\u7f16\u7801\u5c06\u4f1a\u662f\u524d\u9762\u63d0\u5230\u7684 UTF-8\u3002\u66f4\u591a\u4fe1\u606f\u8bf7\u53c2\u9605 PEP 263 \u3002<\/p>\n
                            Unicode\u5c5e\u6027<\/h3>\n
                            Unicode \u89c4\u8303\u5305\u542b\u4e86\u4e00\u4e2a\u7801\u4f4d\u4fe1\u606f\u6570\u636e\u5e93\u3002\u5bf9\u4e8e\u5b9a\u4e49\u7684\u6bcf\u4e00\u4e2a\u7801\u4f4d\uff0c\u90fd\u5305\u542b\u4e86\u5b57\u7b26\u7684\u540d\u79f0\u3001\u7c7b\u522b\u3001\u6570\u503c\uff08\u5bf9\u4e8e\u8868\u793a\u6570\u5b57\u6982\u5ff5\u7684\u5b57\u7b26\uff0c\u5982\u7f57\u9a6c\u6570\u5b57\u3001\u5206\u6570\u5982\u4e09\u5206\u4e4b\u4e00\u548c\u4e94\u5206\u4e4b\u56db\u7b49\uff09\u3002\u8fd8\u6709\u6709\u5173\u663e\u793a\u7684\u5c5e\u6027\uff0c\u6bd4\u5982\u5982\u4f55\u5728\u53cc\u5411\u6587\u672c\u4e2d\u4f7f\u7528\u7801\u4f4d\u3002<\/p>\n
                            \u4ee5\u4e0b\u7a0b\u5e8f\u663e\u793a\u4e86\u51e0\u4e2a\u5b57\u7b26\u7684\u4fe1\u606f\uff0c\u5e76\u6253\u5370\u4e00\u4e2a\u5b57\u7b26\u7684\u6570\u503c\uff1a<\/p>\n
                            \n \n \n \n 
                              \n  \n  \n  \n  
                            1. import<\/span> unicodedata<\/span><\/code><\/li>\n  \n  \n  \n  
                            2. <\/code><\/li>\n  \n  \n  \n  
                            3. u <\/span>=<\/span> chr<\/span>(<\/span>233<\/span>)<\/span> <\/span>+<\/span> chr<\/span>(<\/span>0x0bf2<\/span>)<\/span> <\/span>+<\/span> chr<\/span>(<\/span>3972<\/span>)<\/span> <\/span>+<\/span> chr<\/span>(<\/span>6000<\/span>)<\/span> <\/span>+<\/span> chr<\/span>(<\/span>13231<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                            4. <\/code><\/li>\n  \n  \n  \n  
                            5. for<\/span> i<\/span>,<\/span> c <\/span>in<\/span> enumerate<\/span>(<\/span>u<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                            6.  <\/span>print<\/span>(<\/span>i<\/span>,<\/span> <\/span>'%04x'<\/span> <\/span>%<\/span> ord<\/span>(<\/span>c<\/span>),<\/span> unicodedata<\/span>.<\/span>category<\/span>(<\/span>c<\/span>),<\/span> <\/span>end<\/span>=<\/span>\" \"<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                            7.  <\/span>print<\/span>(<\/span>unicodedata<\/span>.<\/span>name<\/span>(<\/span>c<\/span>))<\/span><\/code><\/li>\n  \n  \n  \n  
                            8. <\/code><\/li>\n  \n  \n  \n  
                            9. # Get numeric value of second character<\/span><\/code><\/li>\n  \n  \n  \n  
                            10. print<\/span>(<\/span>unicodedata<\/span>.<\/span>numeric<\/span>(<\/span>u<\/span>[<\/span>1<\/span>]))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                              \u5f53\u8fd0\u884c\u65f6\uff0c\u8fd9\u5c06\u6253\u5370\u51fa\uff1a<\/p>\n
                              \n \n \n \n 
                                \n  \n  \n  \n  
                              1. 0<\/span> <\/span>00e9<\/span> <\/span>Ll<\/span> LATIN SMALL LETTER E WITH ACUTE<\/span><\/code><\/li>\n  \n  \n  \n  
                              2. 1<\/span> <\/span>0bf2<\/span> <\/span>No<\/span> TAMIL NUMBER ONE THOUSAND<\/span><\/code><\/li>\n  \n  \n  \n  
                              3. 2<\/span> <\/span>0f84<\/span> <\/span>Mn<\/span> TIBETAN MARK HALANTA<\/span><\/code><\/li>\n  \n  \n  \n  
                              4. 3<\/span> <\/span>1770<\/span> <\/span>Lo<\/span> TAGBANWA LETTER SA<\/span><\/code><\/li>\n  \n  \n  \n  
                              5. 4<\/span> <\/span>33af<\/span> <\/span>So<\/span> SQUARE RAD OVER S SQUARED<\/span><\/code><\/li>\n  \n  \n  \n  
                              6. 1000.0<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                \u7c7b\u522b\u4ee3\u7801\u662f\u63cf\u8ff0\u5b57\u7b26\u6027\u8d28\u7684\u4e00\u4e2a\u7f29\u5199\u3002\u5206\u4e3a\u201c\u5b57\u6bcd\u201d\u3001\u201c\u6570\u5b57\u201d\u3001\u201c\u6807\u70b9\u7b26\u53f7\u201d\u6216\u201c\u7b26\u53f7\u201d\u7b49\u7c7b\u522b\uff0c\u800c\u8fd9\u4e9b\u7c7b\u522b\u53c8\u5206\u4e3a\u5b50\u7c7b\u522b\u3002\u5c31\u4ee5\u4e0a\u8f93\u51fa\u7684\u4ee3\u7801\u800c\u8a00\uff0c'Ll'<\/code> \u8868\u793a\u201c\u5b57\u6bcd\uff0c\u5c0f\u5199\u201d\uff0c'No'<\/code> \u8868\u793a\u201c\u6570\u5b57\uff0c\u5176\u4ed6\u201d\uff0c'Mn'<\/code> \u8868\u793a\u201c\u6807\u8bb0\uff0c\u975e\u7a7a\u767d\u7b26\u201d , 'So'<\/code> \u662f\u201c\u7b26\u53f7\uff0c\u5176\u4ed6\u201d\u3002\u6709\u5173\u7c7b\u522b\u4ee3\u7801\u7684\u6e05\u5355\uff0c\u8bf7\u53c2\u9605 Unicode \u5b57\u7b26\u5e93\u6587\u6863 <https:\/\/www.unicode.org\/reports\/tr44\/#General_Category_Values>`_ \u7684\u201c\u901a\u7528\u7c7b\u522b\u503c\u201d\u90e8\u5206\u3002<\/p>\n
                                \u5b57\u7b26\u4e32\u6bd4\u8f83<\/h3>\n
                                Unicode \u8ba9\u5b57\u7b26\u4e32\u7684\u6bd4\u8f83\u53d8\u5f97\u590d\u6742\u4e86\u4e00\u4e9b\uff0c\u56e0\u4e3a\u540c\u4e00\u7ec4\u5b57\u7b26\u53ef\u80fd\u7531\u4e0d\u540c\u7684\u7801\u4f4d\u5e8f\u5217\u7ec4\u6210\u3002\u4f8b\u5982\uff0c\u50cf\u201c\u00ea\u201d\u8fd9\u6837\u7684\u5b57\u6bcd\u53ef\u4ee5\u8868\u793a\u4e3a\u5355\u7801\u4f4d U+00EA\uff0c\u6216\u662f U+0065 U+0302\uff0c\u5373\u201ce\u201d\u7684\u7801\u4f4d\u540e\u8ddf\u201cCOMBINING CIRCUMFLEX ACCENT\u201d\u7684\u7801\u4f4d\u3002\u867d\u7136\u5728\u6253\u5370\u65f6\u4f1a\u4ea7\u751f\u540c\u6837\u7684\u8f93\u51fa\uff0c\u4f46\u4e00\u4e2a\u662f\u957f\u5ea6\u4e3a 1 \u7684\u5b57\u7b26\u4e32\uff0c\u53e6\u4e00\u4e2a\u662f\u957f\u5ea6\u4e3a 2 \u7684\u5b57\u7b26\u4e32\u3002<\/p>\n
                                \u4e00\u79cd\u4e0d\u533a\u5206\u5927\u5c0f\u5199\u6bd4\u8f83\u7684\u5de5\u5177\u662f\u5b57\u7b26\u4e32\u65b9\u6cd5 casefold() \uff0c\u5c06\u6309\u7167 Unicode \u6807\u51c6\u63cf\u8ff0\u7684\u7b97\u6cd5\u5c06\u5b57\u7b26\u4e32\u8f6c\u6362\u4e3a\u4e0d\u533a\u5206\u5927\u5c0f\u5199\u7684\u5f62\u5f0f\u3002\u8be5\u7b97\u6cd5\u5bf9\u8bf8\u5982\u5fb7\u8bed\u5b57\u6bcd\u201c\u00df\u201d\uff08\u4ee3\u7801\u70b9 U+00DF\uff09\u4e4b\u7c7b\u7684\u5b57\u7b26\u8fdb\u884c\u4e86\u7279\u6b8a\u5904\u7406\uff0c\u53d8\u4e3a\u4e00\u5bf9\u5c0f\u5199\u5b57\u6bcd\u201css\u201d\u3002<\/p>\n
                                \n \n \n \n 
                                  \n  \n  \n  \n  
                                1. >>><\/span> street <\/span>=<\/span> <\/span>'G\u00fcrzenichstra\u00dfe'<\/span><\/code><\/li>\n  \n  \n  \n  
                                2. >>><\/span> street<\/span>.<\/span>casefold<\/span>()<\/span><\/code><\/li>\n  \n  \n  \n  
                                3. 'g\u00fcrzenichstrasse'<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                  \u7b2c\u4e8c\u4e2a\u5de5\u5177\u662f unicodedata \u6a21\u5757\u7684 normalize() \u51fd\u6570\uff0c\u5c06\u5b57\u7b26\u4e32\u8f6c\u6362\u4e3a\u51e0\u79cd\u89c4\u8303\u5316\u5f62\u5f0f\u4e4b\u4e00\uff0c\u5176\u4e2d\u540e\u8ddf\u7ec4\u5408\u5b57\u7b26\u7684\u5b57\u6bcd\u5c06\u88ab\u66ff\u6362\u4e3a\u5355\u4e2a\u5b57\u7b26\u3002 normalize()<\/code> \u53ef\u7528\u4e8e\u6267\u884c\u5b57\u7b26\u4e32\u6bd4\u8f83\uff0c\u5373\u4fbf\u4e24\u4e2a\u5b57\u7b26\u4e32\u91c7\u7528\u4e0d\u540c\u7684\u5b57\u7b26\u7ec4\u5408\uff0c\u4e5f\u4e0d\u4f1a\u9519\u8bef\u5730\u62a5\u544a\u4e24\u8005\u4e0d\u76f8\u7b49\uff1a<\/p>\n
                                  \n \n \n \n 
                                    \n  \n  \n  \n  
                                  1. import<\/span> unicodedata<\/span><\/code><\/li>\n  \n  \n  \n  
                                  2. <\/code><\/li>\n  \n  \n  \n  
                                  3. def<\/span> compare_strs<\/span>(<\/span>s1<\/span>,<\/span> s2<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                                  4.  <\/span>def<\/span> NFD<\/span>(<\/span>s<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                                  5.  <\/span>return<\/span> unicodedata<\/span>.<\/span>normalize<\/span>(<\/span>'NFD'<\/span>,<\/span> s<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                  6. <\/code><\/li>\n  \n  \n  \n  
                                  7.  <\/span>return<\/span> NFD<\/span>(<\/span>s1<\/span>)<\/span> <\/span>==<\/span> NFD<\/span>(<\/span>s2<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                  8. <\/code><\/li>\n  \n  \n  \n  
                                  9. single_char <\/span>=<\/span> <\/span>'\u00ea'<\/span><\/code><\/li>\n  \n  \n  \n  
                                  10. multiple_chars <\/span>=<\/span> <\/span>'\\N{LATIN SMALL LETTER E}\\N{COMBINING CIRCUMFLEX ACCENT}'<\/span><\/code><\/li>\n  \n  \n  \n  
                                  11. print<\/span>(<\/span>'length of first string='<\/span>,<\/span> len<\/span>(<\/span>single_char<\/span>))<\/span><\/code><\/li>\n  \n  \n  \n  
                                  12. print<\/span>(<\/span>'length of second string='<\/span>,<\/span> len<\/span>(<\/span>multiple_chars<\/span>))<\/span><\/code><\/li>\n  \n  \n  \n  
                                  13. print<\/span>(<\/span>compare_strs<\/span>(<\/span>single_char<\/span>,<\/span> multiple_chars<\/span>))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                    \u5f53\u8fd0\u884c\u65f6\uff0c\u8fd9\u5c06\u8f93\u51fa\uff1a<\/p>\n
                                    \n \n \n \n 
                                      \n  \n  \n  \n  
                                    1. $ python3 compare<\/span>-<\/span>strs<\/span>.<\/span>py<\/span><\/code><\/li>\n  \n  \n  \n  
                                    2. length of first <\/span>string<\/span>=<\/span> <\/span>1<\/span><\/code><\/li>\n  \n  \n  \n  
                                    3. length of second <\/span>string<\/span>=<\/span> <\/span>2<\/span><\/code><\/li>\n  \n  \n  \n  
                                    4. True<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                      normalize() \u51fd\u6570\u7684\u7b2c\u4e00\u4e2a\u53c2\u6570\u662f\u4e2a\u5b57\u7b26\u4e32\uff0c\u7ed9\u51fa\u6240\u9700\u7684\u89c4\u8303\u5316\u5f62\u5f0f\uff0c\u53ef\u4ee5\u662f\u201cNFC\u201d\u3001\u201cNFKC\u201d\u3001\u201cNFD\u201d\u548c\u201cNFKD\u201d\u4e4b\u4e00\u3002<\/p>\n
                                      Unicode \u6807\u51c6\u8fd8\u8bbe\u5b9a\u4e86\u5982\u4f55\u8fdb\u884c\u4e0d\u533a\u5206\u5927\u5c0f\u5199\u7684\u6bd4\u8f83\uff1a<\/p>\n
                                      \n \n \n \n 
                                        \n  \n  \n  \n  
                                      1. import<\/span> unicodedata<\/span><\/code><\/li>\n  \n  \n  \n  
                                      2. <\/code><\/li>\n  \n  \n  \n  
                                      3. def<\/span> compare_caseless<\/span>(<\/span>s1<\/span>,<\/span> s2<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                                      4.  <\/span>def<\/span> NFD<\/span>(<\/span>s<\/span>):<\/span><\/code><\/li>\n  \n  \n  \n  
                                      5.  <\/span>return<\/span> unicodedata<\/span>.<\/span>normalize<\/span>(<\/span>'NFD'<\/span>,<\/span> s<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                      6. <\/code><\/li>\n  \n  \n  \n  
                                      7.  <\/span>return<\/span> NFD<\/span>(<\/span>NFD<\/span>(<\/span>s1<\/span>).<\/span>casefold<\/span>())<\/span> <\/span>==<\/span> NFD<\/span>(<\/span>NFD<\/span>(<\/span>s2<\/span>).<\/span>casefold<\/span>())<\/span><\/code><\/li>\n  \n  \n  \n  
                                      8. <\/code><\/li>\n  \n  \n  \n  
                                      9. # Example usage<\/span><\/code><\/li>\n  \n  \n  \n  
                                      10. single_char <\/span>=<\/span> <\/span>'\u00ea'<\/span><\/code><\/li>\n  \n  \n  \n  
                                      11. multiple_chars <\/span>=<\/span> <\/span>'\\N{LATIN CAPITAL LETTER E}\\N{COMBINING CIRCUMFLEX ACCENT}'<\/span><\/code><\/li>\n  \n  \n  \n  
                                      12. <\/code><\/li>\n  \n  \n  \n  
                                      13. print<\/span>(<\/span>compare_caseless<\/span>(<\/span>single_char<\/span>,<\/span> multiple_chars<\/span>))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                        \u8fd9\u5c06\u6253\u5370 True<\/code> \u3002\uff08\u4e3a\u4ec0\u4e48 NFD()<\/code> \u4f1a\u88ab\u8c03\u7528\u4e24\u6b21\uff1f\u56e0\u4e3a\u6709\u51e0\u4e2a\u5b57\u7b26\u8ba9 casefold()<\/code> \u8fd4\u56de\u975e\u89c4\u8303\u5316\u7684\u5b57\u7b26\u4e32\uff0c\u6240\u4ee5\u7ed3\u679c\u9700\u8981\u518d\u6b21\u8fdb\u884c\u89c4\u8303\u5316\u3002\u53c2\u89c1 Unicode \u6807\u51c6\u7684 3.13 \u8282 \u7684\u4e00\u4e2a\u8ba8\u8bba\u548c\u793a\u4f8b\u3002\uff09<\/p>\n
                                        Unicode \u6b63\u5219\u8868\u8fbe\u5f0f<\/h3>\n
                                        re \u6a21\u5757\u652f\u6301\u7684\u6b63\u5219\u8868\u8fbe\u5f0f\u53ef\u4ee5\u7528\u5b57\u8282\u4e32\u6216\u5b57\u7b26\u4e32\u7684\u5f62\u5f0f\u63d0\u4f9b\u3002\u6709\u4e00\u4e9b\u7279\u6b8a\u5b57\u7b26\u5e8f\u5217\uff0c\u6bd4\u5982 \\d<\/code> \u548c \\w<\/code> \u5177\u6709\u4e0d\u540c\u7684\u542b\u4e49\uff0c\u5177\u4f53\u53d6\u51b3\u4e8e\u5339\u914d\u6a21\u5f0f\u662f\u4ee5\u5b57\u8282\u4e32\u8fd8\u662f\u5b57\u7b26\u4e32\u5f62\u5f0f\u63d0\u4f9b\u7684\u3002\u4f8b\u5982\uff0c\\d<\/code> \u5c06\u5339\u914d\u5b57\u8282\u4e32\u4e2d\u7684\u5b57\u7b26 [0-9]<\/code> \uff0c\u4f46\u5bf9\u4e8e\u5b57\u7b26\u4e32\u5c06\u4f1a\u5339\u914d 'Nd'<\/code> \u7c7b\u522b\u4e2d\u7684\u4efb\u4f55\u5b57\u7b26\u3002<\/p>\n
                                        \u4e0a\u8ff0\u793a\u4f8b\u4e2d\u7684\u5b57\u7b26\u4e32\u5305\u542b\u4e86\u6cf0\u8bed\u548c\u963f\u62c9\u4f2f\u6570\u5b57\u4e66\u5199\u7684\u6570\u5b57 57\uff1a<\/p>\n
                                        \n \n \n \n 
                                          \n  \n  \n  \n  
                                        1. import<\/span> re<\/span><\/code><\/li>\n  \n  \n  \n  
                                        2. p <\/span>=<\/span> re<\/span>.<\/span>compile<\/span>(<\/span>r<\/span>'\\d+'<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                        3. <\/code><\/li>\n  \n  \n  \n  
                                        4. s <\/span>=<\/span> <\/span>\"Over \\u0e55\\u0e57 57 flavours\"<\/span><\/code><\/li>\n  \n  \n  \n  
                                        5. m <\/span>=<\/span> p<\/span>.<\/span>search<\/span>(<\/span>s<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                        6. print<\/span>(<\/span>repr<\/span>(<\/span>m<\/span>.<\/span>group<\/span>()))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                          \u6267\u884c\u65f6\uff0c\\d+<\/code> \u5c06\u5339\u914d\u4e0a\u6cf0\u8bed\u6570\u5b57\u5e76\u6253\u5370\u51fa\u6765\u3002\u5982\u679c\u5411 compile() \u63d0\u4f9b\u7684\u662f re.ASCII \u6807\u5fd7\uff0c\\d+<\/code> \u5219\u4f1a\u5339\u914d\u5b50\u4e32 \u201c57\u201d\u3002<\/p>\n
                                          \u7c7b\u4f3c\u5730\uff0c\\w<\/code> \u5c06\u5339\u914d\u591a\u79cd Unicode \u5b57\u7b26\uff0c\u4f46\u5bf9\u4e8e\u5b57\u8282\u4e32\u5219\u53ea\u4f1a\u5339\u914d [a-zA-Z0-9_]<\/code> \uff0c\u5982\u679c\u6307\u5b9a re.ASCII\uff0c \\s `` \u5c06\u5339\u914d Unicode \u7a7a\u767d\u7b26\u6216 ``[ \\t\\n\\r\\f\\v]<\/code>\u3002<\/p>\n
                                          \u53c2\u8003\u6587\u732e<\/h3>\n
                                          \u5173\u4e8e Python \u7684 Unicode \u652f\u6301\uff0c\u5176\u4ed6\u8fd8\u6709\u4e00\u4e9b\u5f88\u597d\u7684\u8ba8\u8bba\uff1a<\/p>\n
                                            \n
                                          • \n
                                            Processing Text Files in Python 3, by Nick Coghlan.<\/p>\n<\/li>\n
                                          • \n
                                            \u5b9e\u7528\u7684 Unicode\uff0cNed Batchelder \u5728 PyCon 2012 \u4e0a\u7684\u6f14\u793a\u3002<\/p>\n<\/li>\n<\/ul>\n
                                            str \u7c7b\u578b\u5728 Python \u5e93\u53c2\u8003\u6587\u6863 \u6587\u672c\u5e8f\u5217\u7c7b\u578b \u2014- str \u4e2d\u6709\u4ecb\u7ecd\u3002<\/p>\n
                                            unicodedata \u6a21\u5757\u7684\u6587\u6863<\/p>\n
                                            codecs \u6a21\u5757\u7684\u6587\u6863<\/p>\n
                                            Marc-Andr\u00e9 Lemburg \u5728 EuroPython 2002 \u4e0a\u505a\u4e86\u4e00\u4e2a\u9898\u4e3a\u201cPython \u548c Unicode\u201d\uff08PDF \u5e7b\u706f\u7247\uff09<https:\/\/downloads.egenix.com\/python\/Unicode-EPC2002-Talk.pdf>`_ \u7684\u6f14\u793a\u6587\u7a3f\u3002\u8be5\u5e7b\u706f\u7247\u5f88\u597d\u5730\u6982\u62ec\u4e86 Python 2 \u7684 Unicode \u529f\u80fd\u8bbe\u8ba1\uff08\u5176\u4e2d Unicode \u5b57\u7b26\u4e32\u7c7b\u578b\u79f0\u4e3a unicode<\/code>\uff0c\u6587\u5b57\u4ee5 u<\/code> \u5f00\u5934\uff09\u3002<\/p>\n
                                            Unicode \u6570\u636e\u7684\u8bfb\u5199<\/h2>\n
                                            \u65e2\u7136\u5904\u7406 Unicode \u6570\u636e\u7684\u4ee3\u7801\u5199\u597d\u4e86\uff0c\u4e0b\u4e00\u4e2a\u95ee\u9898\u5c31\u662f\u8f93\u5165\/\u8f93\u51fa\u4e86\u3002\u5982\u4f55\u5c06 Unicode \u5b57\u7b26\u4e32\u8bfb\u5165\u7a0b\u5e8f\uff0c\u5982\u4f55\u5c06 Unicode \u8f6c\u6362\u4e3a\u9002\u4e8e\u5b58\u50a8\u6216\u4f20\u8f93\u7684\u5f62\u5f0f\u5462\uff1f<\/p>\n
                                            \u6839\u636e\u8f93\u5165\u6e90\u548c\u8f93\u51fa\u76ee\u6807\u7684\u4e0d\u540c\uff0c\u6216\u8bb8\u4ec0\u4e48\u90fd\u4e0d\u7528\u5e72\uff1b\u8bf7\u68c0\u67e5\u4e00\u4e0b\u5e94\u7528\u7a0b\u5e8f\u7528\u5230\u7684\u5e93\u662f\u5426\u539f\u751f\u652f\u6301 Unicode\u3002\u4f8b\u5982\uff0cXML \u89e3\u6790\u5668\u5f80\u5f80\u4f1a\u8fd4\u56de Unicode \u6570\u636e\u3002\u8bb8\u591a\u5173\u7cfb\u6570\u636e\u5e93\u7684\u5b57\u6bb5\u4e5f\u652f\u6301 Unicode \u503c\uff0c\u5e76\u4e14 SQL \u67e5\u8be2\u4e5f\u80fd\u8fd4\u56de Unicode \u503c\u3002<\/p>\n
                                            \u5728\u5199\u5165\u78c1\u76d8\u6216\u901a\u8fc7\u5957\u63a5\u5b57\u53d1\u9001\u4e4b\u524d\uff0cUnicode \u6570\u636e\u901a\u5e38\u8981\u8f6c\u6362\u4e3a\u7279\u5b9a\u7684\u7f16\u7801\u3002\u53ef\u4ee5\u81ea\u5df1\u5b8c\u6210\u6240\u6709\u5de5\u4f5c\uff1a\u6253\u5f00\u4e00\u4e2a\u6587\u4ef6\uff0c\u4ece\u4e2d\u8bfb\u53d6\u4e00\u4e2a 8 \u4f4d\u5b57\u8282\u5bf9\u8c61\uff0c\u7136\u540e\u7528 bytes.decode(encoding)<\/code> \u5bf9\u5b57\u8282\u4e32\u8fdb\u884c\u8f6c\u6362\u3002\u4f46\u662f\uff0c\u4e0d\u63a8\u8350\u91c7\u7528\u8fd9\u79cd\u5168\u4eba\u5de5\u7684\u65b9\u6848\u3002<\/p>\n
                                            \u7f16\u7801\u7684\u591a\u5b57\u8282\u7279\u6027\u5c31\u662f\u4e00\u4e2a\u96be\u9898\uff1b \u4e00\u4e2a Unicode \u5b57\u7b26\u53ef\u4ee5\u7528\u51e0\u4e2a\u5b57\u8282\u8868\u793a\u3002 \u5982\u679c\u8981\u4ee5\u4efb\u610f\u5927\u5c0f\u7684\u5757\uff08\u4f8b\u5982 1024 \u6216 4096 \u5b57\u8282\uff09\u8bfb\u53d6\u6587\u4ef6\uff0c\u90a3\u4e48\u5728\u5757\u7684\u672b\u5c3e\u53ef\u80fd\u53ea\u8bfb\u5230\u67d0\u4e2a Unicode \u5b57\u7b26\u7684\u90e8\u5206\u5b57\u8282\uff0c\u8fd9\u5c31\u9700\u8981\u7f16\u5199\u9519\u8bef\u5904\u7406\u4ee3\u7801\u3002 \u6709\u4e00\u79cd\u89e3\u51b3\u65b9\u6848\u662f\u5c06\u6574\u4e2a\u6587\u4ef6\u8bfb\u5165\u5185\u5b58\uff0c\u7136\u540e\u8fdb\u884c\u89e3\u7801\uff0c\u4f46\u8fd9\u6837\u5c31\u6ca1\u6cd5\u5904\u7406\u5f88\u5927\u7684\u6587\u4ef6\u4e86\uff1b\u82e5\u8981\u8bfb\u53d6 2 GB \u7684\u6587\u4ef6\uff0c\u5c31\u9700\u8981 2 GB \u7684 RAM\u3002\uff08\u5176\u5b9e\u9700\u8981\u7684\u5185\u5b58\u4f1a\u66f4\u591a\u4e9b\uff0c\u56e0\u4e3a\u81f3\u5c11\u6709\u4e00\u6bb5\u65f6\u95f4\u9700\u8981\u5728\u5185\u5b58\u4e2d\u540c\u65f6\u5b58\u653e\u5df2\u7f16\u7801\u5b57\u7b26\u4e32\u53ca\u5176 Unicode \u7248\u672c\u3002\uff09<\/p>\n
                                            \u89e3\u51b3\u65b9\u6848\u662f\u5229\u7528\u5e95\u5c42\u89e3\u7801\u63a5\u53e3\u53bb\u6355\u83b7\u7f16\u7801\u5e8f\u5217\u4e0d\u5b8c\u6574\u7684\u60c5\u51b5\u3002\u8fd9\u90e8\u5206\u4ee3\u7801\u5df2\u7ecf\u662f\u73b0\u6210\u7684\uff1a\u5185\u7f6e\u51fd\u6570 open() \u53ef\u4ee5\u8fd4\u56de\u4e00\u4e2a\u6587\u4ef6\u7c7b\u7684\u5bf9\u8c61\uff0c\u8be5\u5bf9\u8c61\u8ba4\u4e3a\u6587\u4ef6\u7684\u5185\u5bb9\u91c7\u7528\u6307\u5b9a\u7684\u7f16\u7801\uff0cread() \u548c write() \u7b49\u65b9\u6cd5\u63a5\u53d7 Unicode \u53c2\u6570\u3002\u53ea\u8981\u7528 open() \u7684 encoding<\/em> \u548c errors<\/em> \u53c2\u6570\u5373\u53ef\uff0c\u53c2\u6570\u91ca\u4e49\u540c str.encode() \u548c bytes.decode() \u3002<\/p>\n
                                            \u56e0\u6b64\u4ece\u6587\u4ef6\u8bfb\u53d6 Unicode \u5c31\u6bd4\u8f83\u7b80\u5355\u4e86\uff1a<\/p>\n
                                            \n \n \n \n 
                                              \n  \n  \n  \n  
                                            1. with<\/span> open<\/span>(<\/span>'unicode.txt'<\/span>,<\/span> encoding<\/span>=<\/span>'utf-8'<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
                                            2.  <\/span>for<\/span> line <\/span>in<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
                                            3.  <\/span>print<\/span>(<\/span>repr<\/span>(<\/span>line<\/span>))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                              \u4e5f\u53ef\u4ee5\u5728\u66f4\u65b0\u6a21\u5f0f\u4e0b\u6253\u5f00\u6587\u4ef6\uff0c\u4ee5\u4fbf\u540c\u65f6\u8bfb\u53d6\u548c\u5199\u5165\uff1a<\/p>\n
                                              \n \n \n \n 
                                                \n  \n  \n  \n  
                                              1. with<\/span> open<\/span>(<\/span>'test'<\/span>,<\/span> encoding<\/span>=<\/span>'utf-8'<\/span>,<\/span> mode<\/span>=<\/span>'w+'<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
                                              2.  f<\/span>.<\/span>write<\/span>(<\/span>'\\u4500 blah blah blah\\n'<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                              3.  f<\/span>.<\/span>seek<\/span>(<\/span>0<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                              4.  <\/span>print<\/span>(<\/span>repr<\/span>(<\/span>f<\/span>.<\/span>readline<\/span>()[:<\/span>1<\/span>]))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                                Unicode \u5b57\u7b26 U+FEFF<\/code> \u7528\u4f5c\u5b57\u8282\u987a\u5e8f\u6807\u8bb0\uff08BOM\uff09\uff0c\u901a\u5e38\u4f5c\u4e3a\u6587\u4ef6\u7684\u7b2c\u4e00\u4e2a\u5b57\u7b26\u5199\u5165\uff0c\u4ee5\u5e2e\u52a9\u81ea\u52a8\u68c0\u6d4b\u6587\u4ef6\u7684\u5b57\u8282\u987a\u5e8f\u3002\u67d0\u4e9b\u7f16\u7801\uff08\u4f8b\u5982 UTF-16\uff09\u671f\u671b\u5728\u6587\u4ef6\u5f00\u5934\u51fa\u73b0 BOM\uff1b\u5f53\u91c7\u7528\u8fd9\u79cd\u7f16\u7801\u65f6\uff0cBOM \u5c06\u81ea\u52a8\u4f5c\u4e3a\u7b2c\u4e00\u4e2a\u5b57\u7b26\u5199\u5165\uff0c\u5e76\u5728\u8bfb\u53d6\u6587\u4ef6\u65f6\u4f1a\u9759\u9ed8\u5220\u9664\u3002\u8fd9\u4e9b\u7f16\u7801\u6709\u591a\u79cd\u53d8\u4f53\uff0c\u4f8b\u5982\u7528\u4e8e little-endian \u548c big-endian \u7f16\u7801\u7684 \u201cutf-16-le\u201d \u548c \u201cutf-16-be\u201d\uff0c\u4f1a\u6307\u5b9a\u4e00\u79cd\u7279\u5b9a\u7684\u5b57\u8282\u987a\u5e8f\u5e76\u4e14\u4e0d\u4f1a\u5ffd\u7565 BOM\u3002<\/p>\n
                                                \u5728\u67d0\u4e9b\u5730\u533a\uff0c\u4e60\u60ef\u5728 UTF-8 \u7f16\u7801\u6587\u4ef6\u7684\u5f00\u5934\u7528\u4e0a\u201cBOM\u201d\uff1b\u6b64\u540d\u79f0\u5177\u6709\u8bef\u5bfc\u6027\uff0c\u56e0\u4e3a UTF-8 \u4e0e\u5b57\u8282\u987a\u5e8f\u65e0\u5173\u3002\u6b64\u6807\u8bb0\u53ea\u662f\u58f0\u660e\u8be5\u6587\u4ef6\u4ee5 UTF-8 \u7f16\u7801\u3002\u8981\u8bfb\u53d6\u6b64\u7c7b\u6587\u4ef6\uff0c\u8bf7\u4f7f\u7528\u201cutf-8-sig\u201d\u7f16\u89e3\u7801\u5668\u81ea\u52a8\u5ffd\u7565\u6b64\u6807\u8bb0\u3002<\/p>\n
                                                Unicode \u6587\u4ef6\u540d<\/h3>\n
                                                \u5f53\u4eca\u5927\u591a\u6570\u64cd\u4f5c\u7cfb\u7edf\u90fd\u652f\u6301\u5305\u542b\u4efb\u610f Unicode \u5b57\u7b26\u7684\u6587\u4ef6\u540d\u3002 \u901a\u5e38\u8fd9\u662f\u901a\u8fc7\u5c06 Unicode \u5b57\u7b26\u4e32\u8f6c\u6362\u4e3a\u67d0\u79cd\u6839\u636e\u5177\u4f53\u7cfb\u7edf\u800c\u5b9a\u7684\u7f16\u7801\u683c\u5f0f\u6765\u5b9e\u73b0\u7684\u3002 \u5982\u4eca\u7684 Python \u503e\u5411\u4e8e\u4f7f\u7528 UTF-8\uff1aMacOS \u4e0a\u7684 Python \u5df2\u7ecf\u5728\u591a\u4e2a\u7248\u672c\u4e2d\u4f7f\u7528\u4e86 UTF-8\uff0c\u800c Python 3.6 \u4e5f\u5df2\u5728 Windows \u4e0a\u6539\u7528\u4e86 UTF-8\u3002 \u5728 Unix \u7cfb\u7edf\u4e2d\uff0c\u5c06\u53ea\u6709\u4e00\u4e2a \u6587\u4ef6\u7cfb\u7edf\u7f16\u7801\u683c\u5f0f\u3002 \u5982\u679c\u4f60\u5df2\u8bbe\u7f6e\u4e86 LANG<\/code> \u6216 LC_CTYPE<\/code> \u73af\u5883\u53d8\u91cf\u7684\u8bdd\uff1b\u5982\u679c\u672a\u8bbe\u7f6e\uff0c\u5219\u9ed8\u8ba4\u7f16\u7801\u683c\u5f0f\u8fd8\u662f UTF-8\u3002<\/p>\n
                                                sys.getfilesystemencoding() \u51fd\u6570\u5c06\u8fd4\u56de\u8981\u5728\u5f53\u524d\u7cfb\u7edf\u91c7\u7528\u7684\u7f16\u7801\uff0c\u82e5\u60f3\u624b\u52a8\u8fdb\u884c\u7f16\u7801\u65f6\u5373\u53ef\u7528\u5230\uff0c\u4f46\u65e0\u9700\u591a\u8651\u3002\u5728\u6253\u5f00\u6587\u4ef6\u8fdb\u884c\u8bfb\u5199\u65f6\uff0c\u901a\u5e38\u53ea\u9700\u63d0\u4f9b Unicode \u5b57\u7b26\u4e32\u4f5c\u4e3a\u6587\u4ef6\u540d\uff0c\u4f1a\u81ea\u52a8\u8f6c\u6362\u4e3a\u5408\u9002\u7684\u7f16\u7801\u683c\u5f0f\uff1a<\/p>\n
                                                \n \n \n \n 
                                                  \n  \n  \n  \n  
                                                1. filename <\/span>=<\/span> <\/span>'filename\\u4500abc'<\/span><\/code><\/li>\n  \n  \n  \n  
                                                2. with<\/span> open<\/span>(<\/span>filename<\/span>,<\/span> <\/span>'w'<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
                                                3.  f<\/span>.<\/span>write<\/span>(<\/span>'blah\\n'<\/span>)<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                                  os \u6a21\u5757\u4e2d\u7684\u51fd\u6570\u4e5f\u80fd\u63a5\u53d7 Unicode \u6587\u4ef6\u540d\uff0c\u5982 os.stat() \u3002<\/p>\n
                                                  os.listdir() \u51fd\u6570\u8fd4\u56de\u6587\u4ef6\u540d\uff0c\u8fd9\u5f15\u53d1\u4e86\u4e00\u4e2a\u95ee\u9898\uff1a\u5b83\u5e94\u8be5\u8fd4\u56de\u6587\u4ef6\u540d\u7684 Unicode \u7248\u672c\uff0c\u8fd8\u662f\u5e94\u8be5\u8fd4\u56de\u5305\u542b\u5df2\u7f16\u7801\u7248\u672c\u7684\u5b57\u8282\u4e32\uff1f \u8fd9\u4e24\u8005 os.listdir() \u90fd\u80fd\u505a\u5230\uff0c\u5177\u4f53\u53d6\u51b3\u4e8e\u4f60\u7ed9\u51fa\u7684\u76ee\u5f55\u8def\u5f84\u662f\u5b57\u8282\u4e32\u8fd8\u662f Unicode \u5b57\u7b26\u4e32\u5f62\u5f0f\u7684\u3002 \u5982\u679c\u4f60\u4f20\u5165\u4e00\u4e2a Unicode \u5b57\u7b26\u4e32\u4f5c\u4e3a\u8def\u5f84\uff0c\u6587\u4ef6\u540d\u5c06\u4f7f\u7528\u6587\u4ef6\u7cfb\u7edf\u7684\u7f16\u7801\u683c\u5f0f\u8fdb\u884c\u89e3\u7801\u5e76\u8fd4\u56de\u4e00\u4e2a Unicode \u5b57\u7b26\u4e32\u5217\u8868\uff0c\u800c\u4f20\u5165\u4e00\u4e2a\u5b57\u8282\u4e32\u5f62\u5f0f\u7684\u8def\u5f84\u5219\u5c06\u8fd4\u56de\u5b57\u8282\u4e32\u5f62\u5f0f\u7684\u6587\u4ef6\u540d\u3002 \u4f8b\u5982\uff0c\u5047\u5b9a\u9ed8\u8ba4 \u6587\u4ef6\u7cfb\u7edf\u7f16\u7801 \u4e3a UTF-8\uff0c\u8fd0\u884c\u4ee5\u4e0b\u7a0b\u5e8f:<\/p>\n
                                                  \n \n \n \n 
                                                    \n  \n  \n  \n  
                                                  1. fn <\/span>=<\/span> <\/span>'filename\\u4500abc'<\/span><\/code><\/li>\n  \n  \n  \n  
                                                  2. f <\/span>=<\/span> open<\/span>(<\/span>fn<\/span>,<\/span> <\/span>'w'<\/span>)<\/span><\/code><\/li>\n  \n  \n  \n  
                                                  3. f<\/span>.<\/span>close<\/span>()<\/span><\/code><\/li>\n  \n  \n  \n  
                                                  4. <\/code><\/li>\n  \n  \n  \n  
                                                  5. import<\/span> os<\/span><\/code><\/li>\n  \n  \n  \n  
                                                  6. print<\/span>(<\/span>os<\/span>.<\/span>listdir<\/span>(<\/span>b<\/span>'.'<\/span>))<\/span><\/code><\/li>\n  \n  \n  \n  
                                                  7. print<\/span>(<\/span>os<\/span>.<\/span>listdir<\/span>(<\/span>'.'<\/span>))<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                                    \u5c06\u4ea7\u751f\u4ee5\u4e0b\u8f93\u51fa\uff1a<\/p>\n
                                                    \n \n \n \n 
                                                      \n  \n  \n  \n  
                                                    1. $ python listdir<\/span>-<\/span>test<\/span>.<\/span>py<\/span><\/code><\/li>\n  \n  \n  \n  
                                                    2. [<\/span>b<\/span>'filename\\xe4\\x94\\x80abc'<\/span>,<\/span> <\/span>...]<\/span><\/code><\/li>\n  \n  \n  \n  
                                                    3. [<\/span>'filename\\u4500abc'<\/span>,<\/span> <\/span>...]<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                                      \u7b2c\u4e00\u4e2a\u5217\u8868\u5305\u542b UTF-8 \u7f16\u7801\u7684\u6587\u4ef6\u540d\uff0c\u7b2c\u4e8c\u4e2a\u5217\u8868\u5219\u5305\u542b Unicode \u7248\u672c\u7684\u3002<\/p>\n
                                                      \u8bf7\u6ce8\u610f\uff0c\u5927\u591a\u65f6\u5019\u5e94\u8be5\u575a\u6301\u7528\u8fd9\u4e9b API \u5904\u7406 Unicode\u3002\u5b57\u8282\u4e32 API \u5e94\u8be5\u4ec5\u7528\u4e8e\u53ef\u80fd\u5b58\u5728\u4e0d\u53ef\u89e3\u7801\u6587\u4ef6\u540d\u7684\u7cfb\u7edf\uff1b\u73b0\u5728\u51e0\u4e4e\u4ec5\u5269 Unix \u7cfb\u7edf\u4e86\u3002<\/p>\n
                                                      \u8bc6\u522b Unicode \u7684\u7f16\u7a0b\u6280\u5de7<\/h3>\n
                                                      \u672c\u8282\u63d0\u4f9b\u4e86\u4e00\u4e9b\u5173\u4e8e\u7f16\u5199 Unicode \u5904\u7406\u8f6f\u4ef6\u7684\u5efa\u8bae\u3002<\/p>\n
                                                      \u6700\u91cd\u8981\u7684\u6280\u5de7\u5982\u4e0b\uff1a<\/p>\n
                                                      \n
                                                      \u7a0b\u5e8f\u5e94\u53ea\u5728\u5185\u90e8\u5904\u7406 Unicode \u5b57\u7b26\u4e32\uff0c\u5c3d\u5feb\u5bf9\u8f93\u5165\u6570\u636e\u8fdb\u884c\u89e3\u7801\uff0c\u5e76\u53ea\u5728\u6700\u540e\u5bf9\u8f93\u51fa\u8fdb\u884c\u7f16\u7801\u3002<\/p>\n<\/blockquote>\n
                                                      \u5982\u679c\u5c1d\u8bd5\u7f16\u5199\u7684\u5904\u7406\u51fd\u6570\u5bf9 Unicode \u548c\u5b57\u8282\u4e32\u5f62\u5f0f\u7684\u5b57\u7b26\u4e32\u90fd\u80fd\u63a5\u53d7\uff0c\u5c31\u4f1a\u53d1\u73b0\u7ec4\u5408\u4f7f\u7528\u4e24\u79cd\u4e0d\u540c\u7c7b\u578b\u7684\u5b57\u7b26\u4e32\u65f6\uff0c\u5bb9\u6613\u4ea7\u751f\u5dee\u9519\u3002\u6ca1\u529e\u6cd5\u505a\u5230\u81ea\u52a8\u7f16\u7801\u6216\u89e3\u7801\uff1a\u5982\u679c\u6267\u884c str + bytes<\/code>\uff0c\u5219\u4f1a\u89e6\u53d1 TypeError\u3002<\/p>\n
                                                      \u5f53\u8981\u4f7f\u7528\u7684\u6570\u636e\u6765\u81ea Web \u6d4f\u89c8\u5668\u6216\u5176\u4ed6\u4e0d\u53d7\u4fe1\u6765\u6e90\u65f6\uff0c\u5e38\u7528\u6280\u672f\u662f\u5728\u7528\u8be5\u5b57\u7b26\u4e32\u751f\u6210\u547d\u4ee4\u884c\u4e4b\u524d\uff0c\u6216\u8981\u5b58\u5165\u6570\u636e\u5e93\u4e4b\u524d\uff0c\u5148\u68c0\u67e5\u5b57\u7b26\u4e32\u4e2d\u662f\u5426\u5305\u542b\u975e\u6cd5\u5b57\u7b26\u3002\u8bf7\u4ed4\u7ec6\u68c0\u67e5\u89e3\u7801\u540e\u7684\u5b57\u7b26\u4e32\uff0c\u800c\u4e0d\u662f\u7f16\u7801\u683c\u5f0f\u7684\u5b57\u8282\u4e32\u6570\u636e\uff1b\u6709\u4e9b\u7f16\u7801\u53ef\u80fd\u5177\u5907\u4e00\u4e9b\u6709\u8da3\u7684\u7279\u6027\uff0c\u4f8b\u5982\u4e0e ASCII \u4e0d\u662f\u4e00\u4e00\u5bf9\u5e94\u6216\u4e0d\u5b8c\u5168\u517c\u5bb9\u3002\u5982\u679c\u8f93\u5165\u6570\u636e\u8fd8\u6307\u5b9a\u4e86\u7f16\u7801\u683c\u5f0f\uff0c\u5219\u5c24\u5176\u5982\u6b64\uff0c\u56e0\u4e3a\u653b\u51fb\u8005\u53ef\u4ee5\u9009\u62e9\u4e00\u79cd\u5de7\u5999\u7684\u65b9\u5f0f\u5c06\u6076\u610f\u6587\u672c\u9690\u85cf\u5728\u7ecf\u8fc7\u7f16\u7801\u7684\u5b57\u8282\u6d41\u4e2d\u3002<\/p>\n
                                                      \u5728\u6587\u4ef6\u7f16\u7801\u683c\u5f0f\u4e4b\u95f4\u8fdb\u884c\u8f6c\u6362<\/h4>\n
                                                      StreamRecoder \u7c7b\u53ef\u4ee5\u5728\u4e24\u79cd\u7f16\u7801\u4e4b\u95f4\u900f\u660e\u5730\u8fdb\u884c\u8f6c\u6362\uff0c\u53c2\u6570\u4e3a\u7f16\u7801\u683c\u5f0f\u4e3a #1 \u7684\u6570\u636e\u6d41\uff0c\u8868\u73b0\u884c\u4e3a\u5219\u662f\u7f16\u7801\u683c\u5f0f\u4e3a #2 \u7684\u6570\u636e\u6d41\u3002<\/p>\n
                                                      \u5047\u8bbe\u8f93\u5165\u6587\u4ef6 f<\/em> \u91c7\u7528 Latin-1 \u7f16\u7801\u683c\u5f0f\uff0c\u5373\u53ef\u7528 StreamRecoder \u5305\u88c5\u540e\u8fd4\u56de UTF-8 \u7f16\u7801\u7684\u5b57\u8282\u4e32\uff1a<\/p>\n
                                                      \n \n \n \n 
                                                        \n  \n  \n  \n  
                                                      1. new_f <\/span>=<\/span> codecs<\/span>.<\/span>StreamRecoder<\/span>(<\/span>f<\/span>,<\/span><\/code><\/li>\n  \n  \n  \n  
                                                      2.  <\/span># en\/decoder: used by read() to encode its results and<\/span><\/code><\/li>\n  \n  \n  \n  
                                                      3.  <\/span># by write() to decode its input.<\/span><\/code><\/li>\n  \n  \n  \n  
                                                      4.  codecs<\/span>.<\/span>getencoder<\/span>(<\/span>'utf-8'<\/span>),<\/span> codecs<\/span>.<\/span>getdecoder<\/span>(<\/span>'utf-8'<\/span>),<\/span><\/code><\/li>\n  \n  \n  \n  
                                                      5. <\/code><\/li>\n  \n  \n  \n  
                                                      6.  <\/span># reader\/writer: used to read and write to the stream.<\/span><\/code><\/li>\n  \n  \n  \n  
                                                      7.  codecs<\/span>.<\/span>getreader<\/span>(<\/span>'latin-1'<\/span>),<\/span> codecs<\/span>.<\/span>getwriter<\/span>(<\/span>'latin-1'<\/span>)<\/span> <\/span>)<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                                        \u7f16\u7801\u683c\u5f0f\u672a\u77e5\u7684\u6587\u4ef6<\/h4>\n
                                                        \u82e5\u9700\u5bf9\u6587\u4ef6\u8fdb\u884c\u4fee\u6539\uff0c\u4f46\u4e0d\u77e5\u9053\u6587\u4ef6\u7684\u7f16\u7801\uff0c\u90a3\u8be5\u600e\u4e48\u529e\u5462\uff1f\u5982\u679c\u5df2\u77e5\u7f16\u7801\u683c\u5f0f\u4e0e ASCII \u517c\u5bb9\uff0c\u5e76\u4e14\u53ea\u60f3\u67e5\u770b\u6216\u4fee\u6539 ASCII \u90e8\u5206\uff0c\u5219\u53ef\u5229\u7528 surrogateescape<\/code> \u9519\u8bef\u5904\u7406 handler \u6253\u5f00\u6587\u4ef6\uff1a<\/p>\n
                                                        \n \n \n \n 
                                                          \n  \n  \n  \n  
                                                        1. with<\/span> open<\/span>(<\/span>fname<\/span>,<\/span> <\/span>'r'<\/span>,<\/span> encoding<\/span>=<\/span>\"ascii\"<\/span>,<\/span> errors<\/span>=<\/span>\"surrogateescape\"<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
                                                        2.  data <\/span>=<\/span> f<\/span>.<\/span>read<\/span>()<\/span><\/code><\/li>\n  \n  \n  \n  
                                                        3. <\/code><\/li>\n  \n  \n  \n  
                                                        4. # make changes to the string 'data'<\/span><\/code><\/li>\n  \n  \n  \n  
                                                        5. <\/code><\/li>\n  \n  \n  \n  
                                                        6. with<\/span> open<\/span>(<\/span>fname <\/span>+<\/span> <\/span>'.new'<\/span>,<\/span> <\/span>'w'<\/span>,<\/span><\/code><\/li>\n  \n  \n  \n  
                                                        7.  encoding<\/span>=<\/span>\"ascii\"<\/span>,<\/span> errors<\/span>=<\/span>\"surrogateescape\"<\/span>)<\/span> <\/span>as<\/span> f<\/span>:<\/span><\/code><\/li>\n  \n  \n  \n  
                                                        8.  f<\/span>.<\/span>write<\/span>(<\/span>data<\/span>)<\/span><\/code><\/li>\n \n \n \n <\/ol><\/pre>\n
                                                          surrogateescape<\/code> \u9519\u8bef\u5904\u7406 handler \u4f1a\u628a\u6240\u6709\u975e ASCII \u5b57\u8282\u89e3\u7801\u4e3a U+DC80 \u81f3 U+DCFF \u8fd9\u4e00\u7279\u6b8a\u8303\u56f4\u7684\u7801\u4f4d\u3002\u5f53 surrogateescape<\/code> \u9519\u8bef\u5904\u7406 handler\u7528\u4e8e\u6570\u636e\u7f16\u7801\u5e76\u56de\u5199\u65f6\uff0c\u8fd9\u4e9b\u7801\u4f4d\u5c06\u8f6c\u6362\u56de\u539f\u6837\u3002<\/p>\n
                                                          \u53c2\u8003\u6587\u732e<\/h3>\n
                                                          One section of Mastering Python 3 Input\/Output, a PyCon 2010 talk by David Beazley, discusses text processing and binary data handling.<\/p>\n
                                                          Marc-Andr\u00e9 Lemburg \u6f14\u793a\u7684PDF \u5e7b\u706f\u7247\u201c\u5728 Python \u4e2d\u7f16\u5199\u652f\u6301 Unicode \u7684\u5e94\u7528\u7a0b\u5e8f\u201d \uff0c\u8ba8\u8bba\u4e86\u5b57\u7b26\u7f16\u7801\u95ee\u9898\u4ee5\u53ca\u5982\u4f55\u56fd\u9645\u5316\u548c\u672c\u5730\u5316\u5e94\u7528\u7a0b\u5e8f\u3002\u8fd9\u4e9b\u5e7b\u706f\u7247\u4ec5\u6db5\u76d6 Python 2.x\u3002<\/p>\n
                                                          The Guts of Unicode in Python is a PyCon 2013 talk by Benjamin Peterson that discusses the internal Unicode representation in Python 3.3.<\/p>\n
                                                          \u81f4\u8c22<\/h2>\n
                                                          \u672c\u6587\u521d\u7a3f\u7531 Andrew Kuchling \u64b0\u5199\u3002\u6b64\u540e\uff0cAlexander Belopolsky\u3001Georg Brandl\u3001Andrew Kuchling \u548c Ezio Melotti \u4f5c\u4e86\u8fdb\u4e00\u6b65\u4fee\u8ba2\u3002<\/p>\n
                                                          \u611f\u8c22\u4ee5\u4e0b\u5404\u4f4d\u6307\u51fa\u672c\u6587\u9519\u8bef\u6216\u63d0\u51fa\u5efa\u8bae\uff1a\u00c9ric Araujo\u3001Nicholas Bastin\u3001Nick Coghlan\u3001Marius Gedminas\u3001Kent Johnson\u3001Ken Krugler\u3001Marc-Andr\u00e9 Lemburg\u3001Martin von L\u00f6wis\u3001Terry J. Reedy\u3001Serhiy Storchaka , Eryk Sun, Chad Whitacre, Graham Wideman\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
                                                          Unicode \u6307\u5357 \u53d1\u5e03\u7248\u672c 1.12 \u672c\u6587\u4ecb\u7ecd\u4e86 Python \u5bf9\u8868\u793a\u6587\u672c\u6570\u636e\u7684 Unicode \u89c4\u8303\u7684\u652f […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[182397],"tags":[],"class_list":["post-167907","post","type-post","status-publish","format-standard","hentry","category-python"],"_links":{"self":[{"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/posts\/167907","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/comments?post=167907"}],"version-history":[{"count":0,"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/posts\/167907\/revisions"}],"wp:attachment":[{"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/media?parent=167907"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/categories?post=167907"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/idc.net\/help\/wp-json\/wp\/v2\/tags?post=167907"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}