懂你所需,做你所想
当前位置: 仙桃云科 > 新闻资讯 >

对 url 中含有的中文进行转码操作

发表日期:2018-07-27文章编辑:仙桃云科网络科技有限公司

 

情景说明

android

安卓系统,能够识别中文路径:


https://spdbimg.idoupiao.com/film/my/美国队长3

IOS

苹果系统,不能够识别中文路径:


https://spdbimg.idoupiao.com/film/my/美国队长3

必须将以上路径进行转码为下来形式才可以访问:


https://spdbimg.idoupiao.com/film/my/%E7%BE%8E%E5%9B%BD%E9%98%9F%E9%95%BF3

这和他们各自的战略定位有关系。。。


编码实现

1、先寻找有没有已经实现类似功能的成熟工具。
2、找到 JDK 提供的 URLEncoder 类的 encode 方法基本满足需求。
3、测试是否满足当下需求。
经过测试发现,JDK 提供的 URLEncoder 类的 encode 方法会将 ':' 和 '/' 等字符也进行编码。
4、查找 JDK 提供的 URLEncoder 类的 encode 方法 相关源码。
5、发现可以进行部分改造满足当前需求,重写 URLEncoder 类,并将其设置为私有静态类,多人合作时,防止他人误用。
6、改造完成进行验证型测试。

添加以下两条语句:



    // 排除这两个特殊字符的编码
    dontNeedEncoding.set(':');
    dontNeedEncoding.set('/');

重写后的 URLEncoder 类:



/**
     * Utility class for HTML form encoding. This class contains static methods
     * for converting a String to the <CODE>application/x-www-form-urlencoded</CODE> MIME
     * format. For more information about HTML form encoding, consult the HTML
     * <A HREF="http://www.w3.org/TR/html4/">specification</A>.
     *
     * <p>
     * When encoding a String, the following rules apply:
     *
     * <ul>
     * <li>The alphanumeric characters &quot;{@code a}&quot; through
     *     &quot;{@code z}&quot;, &quot;{@code A}&quot; through
     *     &quot;{@code Z}&quot; and &quot;{@code 0}&quot;
     *     through &quot;{@code 9}&quot; remain the same.
     * <li>The special characters &quot;{@code .}&quot;,
     *     &quot;{@code -}&quot;, &quot;{@code *}&quot;, and
     *     &quot;{@code _}&quot; remain the same.
     * <li>The space character &quot; &nbsp; &quot; is
     *     converted into a plus sign &quot;{@code +}&quot;.
     * <li>All other characters are unsafe and are first converted into
     *     one or more bytes using some encoding scheme. Then each byte is
     *     represented by the 3-character string
     *     &quot;<i>{@code %xy}</i>&quot;, where <i>xy</i> is the
     *     two-digit hexadecimal representation of the byte.
     *     The recommended encoding scheme to use is UTF-8. However,
     *     for compatibility reasons, if an encoding is not specified,
     *     then the default encoding of the platform is used.
     * </ul>
     *
     * <p>
     * For example using UTF-8 as the encoding scheme the string &quot;The
     * string &#252;@foo-bar&quot; would get converted to
     * &quot;The+string+%C3%BC%40foo-bar&quot; because in UTF-8 the character
     * &#252; is encoded as two bytes C3 (hex) and BC (hex), and the
     * character @ is encoded as one byte 40 (hex).
     *
     * @author  Herb Jellinek
     * @since   JDK1.0
     */
    private static class URLEncoder {
        static BitSet dontNeedEncoding;
        static final int caseDiff = ('a' - 'A');
        static String dfltEncName = null;

        static {

        /* The list of characters that are not encoded has been
         * determined as follows:
         *
         * RFC 2396 states:
         * -----
         * Data characters that are allowed in a URI but do not have a
         * reserved purpose are called unreserved.  These include upper
         * and lower case letters, decimal digits, and a limited set of
         * punctuation marks and symbols.
         *
         * unreserved  = alphanum | mark
         *
         * mark        = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"
         *
         * Unreserved characters can be escaped without changing the
         * semantics of the URI, but this should not be done unless the
         * URI is being used in a context that does not allow the
         * unescaped character to appear.
         * -----
         *
         * It appears that both Netscape and Internet Explorer escape
         * all special characters from this list with the exception
         * of "-", "_", ".", "*". While it is not clear why they are
         * escaping the other characters, perhaps it is safest to
         * assume that there might be contexts in which the others
         * are unsafe if not escaped. Therefore, we will use the same
         * list. It is also noteworthy that this is consistent with
         * O'Reilly's "HTML: The Definitive Guide" (page 164).
         *
         * As a last note, Intenet Explorer does not encode the "@"
         * character which is clearly not unreserved according to the
         * RFC. We are being consistent with the RFC in this matter,
         * as is Netscape.
         *
         */

            dontNeedEncoding = new BitSet(256);
            int i;
            for (i = 'a'; i <= 'z'; i++) {
                dontNeedEncoding.set(i);
            }
            for (i = 'A'; i <= 'Z'; i++) {
                dontNeedEncoding.set(i);
            }
            for (i = '0'; i <= '9'; i++) {
                dontNeedEncoding.set(i);
            }
            dontNeedEncoding.set(' '); /* encoding a space to a + is done
                                    * in the encode() method */
            dontNeedEncoding.set('-');
            dontNeedEncoding.set('_');
            dontNeedEncoding.set('.');
            dontNeedEncoding.set('*');

            // 排除这两个特殊字符的编码
            dontNeedEncoding.set(':');
            dontNeedEncoding.set('/');


            dfltEncName = AccessController.doPrivileged(
                    new GetPropertyAction("file.encoding")
            );
        }

        /**
         * You can't call the constructor.
         */
        private URLEncoder() { }

        /**
         * Translates a string into {@code x-www-form-urlencoded}
         * format. This method uses the platform's default encoding
         * as the encoding scheme to obtain the bytes for unsafe characters.
         *
         * @param   s   {@code String} to be translated.
         * @deprecated The resulting string may vary depending on the platform's
         *             default encoding. Instead, use the encode(String,String)
         *             method to specify the encoding.
         * @return  the translated {@code String}.
         */
        @Deprecated
        public static String encode(String s) {

            String str = null;

            try {
                str = encode(s, dfltEncName);
            } catch (UnsupportedEncodingException e) {
                // The system should always have the platform default
            }

            return str;
        }

        /**
         * Translates a string into {@code application/x-www-form-urlencoded}
         * format using a specific encoding scheme. This method uses the
         * supplied encoding scheme to obtain the bytes for unsafe
         * characters.
         * <p>
         * <em><strong>Note:</strong> The <a href=
         * "http://www.w3.org/TR/html40/appendix/notes.html#non-ascii-chars">
         * World Wide Web Consortium Recommendation</a> states that
         * UTF-8 should be used. Not doing so may introduce
         * incompatibilities.</em>
         *
         * @param   s   {@code String} to be translated.
         * @param   enc   The name of a supported
         *    <a href="../lang/package-summary.html#charenc">character
         *    encoding</a>.
         * @return  the translated {@code String}.
         * @exception  UnsupportedEncodingException
         *             If the named encoding is not supported
         * @see URLDecoder#decode(java.lang.String, java.lang.String)
         * @since 1.4
         */
        public static String encode(String s, String enc)
                throws UnsupportedEncodingException {

            boolean needToChange = false;
            StringBuffer out = new StringBuffer(s.length());
            Charset charset;
            CharArrayWriter charArrayWriter = new CharArrayWriter();

            if (enc == null)
                throw new NullPointerException("charsetName");

            try {
                charset = Charset.forName(enc);
            } catch (IllegalCharsetNameException e) {
                throw new UnsupportedEncodingException(enc);
            } catch (UnsupportedCharsetException e) {
                throw new UnsupportedEncodingException(enc);
            }

            for (int i = 0; i < s.length();) {
                int c = (int) s.charAt(i);
                //System.out.println("Examining character: " + c);
                if (dontNeedEncoding.get(c)) {
                    if (c == ' ') {
                        c = '+';
                        needToChange = true;
                    }
                    //System.out.println("Storing: " + c);
                    out.append((char)c);
                    i++;
                } else {
                    // convert to external encoding before hex conversion
                    do {
                        charArrayWriter.write(c);
                    /*
                     * If this character represents the start of a Unicode
                     * surrogate pair, then pass in two characters. It's not
                     * clear what should be done if a bytes reserved in the
                     * surrogate pairs range occurs outside of a legal
                     * surrogate pair. For now, just treat it as if it were
                     * any other character.
                     */
                        if (c >= 0xD800 && c <= 0xDBFF) {
                        /*
                          System.out.println(Integer.toHexString(c)
                          + " is high surrogate");
                        */
                            if ( (i+1) < s.length()) {
                                int d = (int) s.charAt(i+1);
                            /*
                              System.out.println("\tExamining "
                              + Integer.toHexString(d));
                            */
                                if (d >= 0xDC00 && d <= 0xDFFF) {
                                /*
                                  System.out.println("\t"
                                  + Integer.toHexString(d)
                                  + " is low surrogate");
                                */
                                    charArrayWriter.write(d);
                                    i++;
                                }
                            }
                        }
                        i++;
                    } while (i < s.length() && !dontNeedEncoding.get((c = (int) s.charAt(i))));

                    charArrayWriter.flush();
                    String str = new String(charArrayWriter.toCharArray());
                    byte[] ba = str.getBytes(charset);
                    for (int j = 0; j < ba.length; j++) {
                        out.append('%');
                        char ch = Character.forDigit((ba[j] >> 4) & 0xF, 16);
                        // converting to use uppercase letter as part of
                        // the hex value if ch is a letter.
                        if (Character.isLetter(ch)) {
                            ch -= caseDiff;
                        }
                        out.append(ch);
                        ch = Character.forDigit(ba[j] & 0xF, 16);
                        if (Character.isLetter(ch)) {
                            ch -= caseDiff;
                        }
                        out.append(ch);
                    }
                    charArrayWriter.reset();
                    needToChange = true;
                }
            }

            return (needToChange? out.toString() : s);
        }
    }

测试代码:


    @Test
    public void testReplace() throws UnsupportedEncodingException {

        // https://ojjqzz83h.qnssl.com/film/my/美国队长3?imageView2/1/w/140/h/90/interlace/0/q/100
        // https://spdbimg.idoupiao.com/film/my/%E7%BE%8E%E5%9B%BD%E9%98%9F%E9%95%BF3

        String oldImgUrl = "https://spdbimg.idoupiao.com/film/my/美国队长3";

        oldImgUrl = URLEncoder.encode(oldImgUrl, "UTF-8");

//        oldImgUrl = new String(Base64.getEncoder().encode(oldImgUrl.getBytes()));

        String oldDomain = "spdbimg.idoupiao.com";
        String newDomain = "ojjqzz83h.qnssl.com";
        String suffix = "?imageView2/1/w/140/h/90/interlace/0/q/100";

        String newImgUrl = StringUtils.replace(oldImgUrl, oldDomain, newDomain).concat(suffix);

        System.out.println("newImgUrl === " + newImgUrl);

    }
    

完整代码如下:



package com.github.ljmatlight.util;

import org.junit.Test;
import sun.security.action.GetPropertyAction;

import java.io.CharArrayWriter;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.nio.charset.Charset;
import java.nio.charset.IllegalCharsetNameException;
import java.nio.charset.UnsupportedCharsetException;
import java.security.AccessController;
import java.util.BitSet;

/**
 * Description:对 url 中含有的中文进行转码操作
 * <br /> Author:ljmatlight
 */
public class StringTest {


    @Test
    public void testReplace() throws UnsupportedEncodingException {

        // https://ojjqzz83h.qnssl.com/film/my/美国队长3?imageView2/1/w/140/h/90/interlace/0/q/100
        // https://spdbimg.idoupiao.com/film/my/%E7%BE%8E%E5%9B%BD%E9%98%9F%E9%95%BF3

        String oldImgUrl = "https://spdbimg.idoupiao.com/film/my/美国队长3";

        oldImgUrl = URLEncoder.encode(oldImgUrl, "UTF-8");

//        oldImgUrl = new String(Base64.getEncoder().encode(oldImgUrl.getBytes()));

        String oldDomain = "spdbimg.idoupiao.com";
        String newDomain = "ojjqzz83h.qnssl.com";
        String suffix = "?imageView2/1/w/140/h/90/interlace/0/q/100";

        String newImgUrl = StringUtils.replace(oldImgUrl, oldDomain, newDomain).concat(suffix);

        System.out.println("newImgUrl === " + newImgUrl);

    }


    /**
     * Utility class for HTML form encoding. This class contains static methods
     * for converting a String to the <CODE>application/x-www-form-urlencoded</CODE> MIME
     * format. For more information about HTML form encoding, consult the HTML
     * <A HREF="http://www.w3.org/TR/html4/">specification</A>.
     *
     * <p>
     * When encoding a String, the following rules apply:
     *
     * <ul>
     * <li>The alphanumeric characters &quot;{@code a}&quot; through
     *     &quot;{@code z}&quot;, &quot;{@code A}&quot; through
     *     &quot;{@code Z}&quot; and &quot;{@code 0}&quot;
     *     through &quot;{@code 9}&quot; remain the same.
     * <li>The special characters &quot;{@code .}&quot;,
     *     &quot;{@code -}&quot;, &quot;{@code *}&quot;, and
     *     &quot;{@code _}&quot; remain the same.
     * <li>The space character &quot; &nbsp; &quot; is
     *     converted into a plus sign &quot;{@code +}&quot;.
     * <li>All other characters are unsafe and are first converted into
     *     one or more bytes using some encoding scheme. Then each byte is
     *     represented by the 3-character string
     *     &quot;<i>{@code %xy}</i>&quot;, where <i>xy</i> is the
     *     two-digit hexadecimal representation of the byte.
     *     The recommended encoding scheme to use is UTF-8. However,
     *     for compatibility reasons, if an encoding is not specified,
     *     then the default encoding of the platform is used.
     * </ul>
     *
     * <p>
     * For example using UTF-8 as the encoding scheme the string &quot;The
     * string &#252;@foo-bar&quot; would get converted to
     * &quot;The+string+%C3%BC%40foo-bar&quot; because in UTF-8 the character
     * &#252; is encoded as two bytes C3 (hex) and BC (hex), and the
     * character @ is encoded as one byte 40 (hex).
     *
     * @author  Herb Jellinek
     * @since   JDK1.0
     */
    private static class URLEncoder {
        static BitSet dontNeedEncoding;
        static final int caseDiff = ('a' - 'A');
        static String dfltEncName = null;

        static {

        /* The list of characters that are not encoded has been
         * determined as follows:
         *
         * RFC 2396 states:
         * -----
         * Data characters that are allowed in a URI but do not have a
         * reserved purpose are called unreserved.  These include upper
         * and lower case letters, decimal digits, and a limited set of
         * punctuation marks and symbols.
         *
         * unreserved  = alphanum | mark
         *
         * mark        = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"
         *
         * Unreserved characters can be escaped without changing the
         * semantics of the URI, but this should not be done unless the
         * URI is being used in a context that does not allow the
         * unescaped character to appear.
         * -----
         *
         * It appears that both Netscape and Internet Explorer escape
         * all special characters from this list with the exception
         * of "-", "_", ".", "*". While it is not clear why they are
         * escaping the other characters, perhaps it is safest to
         * assume that there might be contexts in which the others
         * are unsafe if not escaped. Therefore, we will use the same
         * list. It is also noteworthy that this is consistent with
         * O'Reilly's "HTML: The Definitive Guide" (page 164).
         *
         * As a last note, Intenet Explorer does not encode the "@"
         * character which is clearly not unreserved according to the
         * RFC. We are being consistent with the RFC in this matter,
         * as is Netscape.
         *
         */

            dontNeedEncoding = new BitSet(256);
            int i;
            for (i = 'a'; i <= 'z'; i++) {
                dontNeedEncoding.set(i);
            }
            for (i = 'A'; i <= 'Z'; i++) {
                dontNeedEncoding.set(i);
            }
            for (i = '0'; i <= '9'; i++) {
                dontNeedEncoding.set(i);
            }
            dontNeedEncoding.set(' '); /* encoding a space to a + is done
                                    * in the encode() method */
            dontNeedEncoding.set('-');
            dontNeedEncoding.set('_');
            dontNeedEncoding.set('.');
            dontNeedEncoding.set('*');

            // 排除这两个特殊字符的编码
            dontNeedEncoding.set(':');
            dontNeedEncoding.set('/');


            dfltEncName = AccessController.doPrivileged(
                    new GetPropertyAction("file.encoding")
            );
        }

        /**
         * You can't call the constructor.
         */
        private URLEncoder() { }

        /**
         * Translates a string into {@code x-www-form-urlencoded}
         * format. This method uses the platform's default encoding
         * as the encoding scheme to obtain the bytes for unsafe characters.
         *
         * @param   s   {@code String} to be translated.
         * @deprecated The resulting string may vary depending on the platform's
         *             default encoding. Instead, use the encode(String,String)
         *             method to specify the encoding.
         * @return  the translated {@code String}.
         */
        @Deprecated
        public static String encode(String s) {

            String str = null;

            try {
                str = encode(s, dfltEncName);
            } catch (UnsupportedEncodingException e) {
                // The system should always have the platform default
            }

            return str;
        }

        /**
         * Translates a string into {@code application/x-www-form-urlencoded}
         * format using a specific encoding scheme. This method uses the
         * supplied encoding scheme to obtain the bytes for unsafe
         * characters.
         * <p>
         * <em><strong>Note:</strong> The <a href=
         * "http://www.w3.org/TR/html40/appendix/notes.html#non-ascii-chars">
         * World Wide Web Consortium Recommendation</a> states that
         * UTF-8 should be used. Not doing so may introduce
         * incompatibilities.</em>
         *
         * @param   s   {@code String} to be translated.
         * @param   enc   The name of a supported
         *    <a href="../lang/package-summary.html#charenc">character
         *    encoding</a>.
         * @return  the translated {@code String}.
         * @exception  UnsupportedEncodingException
         *             If the named encoding is not supported
         * @see URLDecoder#decode(java.lang.String, java.lang.String)
         * @since 1.4
         */
        public static String encode(String s, String enc)
                throws UnsupportedEncodingException {

            boolean needToChange = false;
            StringBuffer out = new StringBuffer(s.length());
            Charset charset;
            CharArrayWriter charArrayWriter = new CharArrayWriter();

            if (enc == null)
                throw new NullPointerException("charsetName");

            try {
                charset = Charset.forName(enc);
            } catch (IllegalCharsetNameException e) {
                throw new UnsupportedEncodingException(enc);
            } catch (UnsupportedCharsetException e) {
                throw new UnsupportedEncodingException(enc);
            }

            for (int i = 0; i < s.length();) {
                int c = (int) s.charAt(i);
                //System.out.println("Examining character: " + c);
                if (dontNeedEncoding.get(c)) {
                    if (c == ' ') {
                        c = '+';
                        needToChange = true;
                    }
                    //System.out.println("Storing: " + c);
                    out.append((char)c);
                    i++;
                } else {
                    // convert to external encoding before hex conversion
                    do {
                        charArrayWriter.write(c);
                    /*
                     * If this character represents the start of a Unicode
                     * surrogate pair, then pass in two characters. It's not
                     * clear what should be done if a bytes reserved in the
                     * surrogate pairs range occurs outside of a legal
                     * surrogate pair. For now, just treat it as if it were
                     * any other character.
                     */
                        if (c >= 0xD800 && c <= 0xDBFF) {
                        /*
                          System.out.println(Integer.toHexString(c)
                          + " is high surrogate");
                        */
                            if ( (i+1) < s.length()) {
                                int d = (int) s.charAt(i+1);
                            /*
                              System.out.println("\tExamining "
                              + Integer.toHexString(d));
                            */
                                if (d >= 0xDC00 && d <= 0xDFFF) {
                                /*
                                  System.out.println("\t"
                                  + Integer.toHexString(d)
                                  + " is low surrogate");
                                */
                                    charArrayWriter.write(d);
                                    i++;
                                }
                            }
                        }
                        i++;
                    } while (i < s.length() && !dontNeedEncoding.get((c = (int) s.charAt(i))));

                    charArrayWriter.flush();
                    String str = new String(charArrayWriter.toCharArray());
                    byte[] ba = str.getBytes(charset);
                    for (int j = 0; j < ba.length; j++) {
                        out.append('%');
                        char ch = Character.forDigit((ba[j] >> 4) & 0xF, 16);
                        // converting to use uppercase letter as part of
                        // the hex value if ch is a letter.
                        if (Character.isLetter(ch)) {
                            ch -= caseDiff;
                        }
                        out.append(ch);
                        ch = Character.forDigit(ba[j] & 0xF, 16);
                        if (Character.isLetter(ch)) {
                            ch -= caseDiff;
                        }
                        out.append(ch);
                    }
                    charArrayWriter.reset();
                    needToChange = true;
                }
            }

            return (needToChange? out.toString() : s);
        }
    }


}


如没特殊注明,文章均为仙桃云科网络原创,转载请注明来自>
返回列表
    下一篇:没有了
相关新闻

对 url 中含有的中文进行转码操...

一般情况下,将带有中文的 url 拷贝到开发工具,开发工具都会有相应的转码(自动转码), 现在大部分的浏览器也可以对含有中文的 url 进行转码(自动转码) 情景说明 andro...

日期:2018-07-27

【视频编解码·学习笔记】6. H.2...

一、准备工作: 新建一个VS工程 SimpleH264Analyzer , 修改工程属性参数- 输出目录: $(SolutionDir)bin\$(Configuration)\ ,工作目录: $(SolutionDir)bin\$(Configurat...

日期:2018-02-12

A/X家FPGA架构及资源评估...

评估对比xilinx以及altera两家FPGA芯片逻辑资源。 首先要说明, 现今FPGA除了常规逻辑资源,还具有很多其他片内资源比如块RAM、DSP单元、高速串行收发器、PLL、ADC等等,用...

日期:2018-02-03

HashMap 源码详细分析(JDK1.8)...

一、概述 本篇文章我们来聊聊大家日常开发中常用的一个集合类 - HashMap 。HashMap 最早出现在 JDK 1.2中,底层基于散列算法实现。HashMap 允许 null 键和 null 值,在计算...

日期:2018-01-22

Angular整合zTree、异步加载节点...

1 前提准备 1.1 新建一个angular4项目 参考博文:点击前往 1.2 去zTree官网下载zTree zTree官网:点击前往 三少使用的版本:点击前往 1.3 参考博客 点击前往01 点击前往02 ...

日期:2018-01-20

记一次高并发场景下.net监控程序...

最近在和小伙伴们做充电与通信程序的架构迁移。迁移前的架构是,通信程序负责接收来自充电集控设备的数据实时数据,通过Thrift调用后端的充电服务,充电服务收到响应后放到...

日期:2017-12-23