欢迎各位兄弟 发布技术文章

这里的技术是共享的

You are here

首页

vim 打开文件末尾带有^M (windows 换行符) 结束 (\r)的解决办法 尖括号 M 有大用 有大大用 有大大大用

星期六, 2018-10-06 08:25 — adminshiping1 

似乎  GB2312 的文件 通过 FTP 的 流格式(而不是 ASCI 格式) 传过去,会发生 ^M 的问题 
但是  UTF-8 的文件不会








sed -i 's/\r$//g' file  // 正确  这里 OK   (\r 就是 ^M ,就是windows的换行符(\r\n)比linux换行符(\n) 多出的 \r ) ( 在vim 中执行 :%s@\r@@g 也行 )




  sed -e 's/.$//g' file // 正确  (点.表示任意单个字词,可能会误删吧)

  sed -i -e 's/.$//g' file // 正确 (-e用不用无所谓)

 sed -i 's/^M//g' file  好多地方说用它,但是我这边测试了不行

  

sed Delete / Remove ^M Carriage Return (Line Feed / CRLF) on Linux or Unix

last updated  in Categories, , , , ,, , 


How can I remove the ^M or ^M (carriage Return / line feed ) from text file using sed under UNIX or Linux operating systems?

A newline is nothing but end of line (EOL). It is a special character or sequence of characters signifying the end of a line of text and the start of a new line. The actual codes representing a newline vary across operating systems. For example CR+LF is used by Microsoft Windows, DOS (MS-DOS, PC DOS, etc.). LF is used by Unix and Unix-like systems including Linux, OS X, FreeBSD and more. The procedure is as follows:




  1. Type the following sed command to delete a carriage Return (CR)

  2. sed 's/\r//' input > output

  3. Type the following sed command to replace a linefeed(LF)

  4. sed ':a;N;$!ba;s/\n//g' input > output

Delete a carriage return (CR) with sed command

The substitute command syntax is as follows (to get ^M type CTRL+V followed by CTRL+M i.e. don’t just type the carat symbol and a capital M. It will not work):

sed -e 's/^M//g' input
sed -e 's/^M//g' input > output
# gnu sed syntax
sed -i 's/^M//g' input
# replace it with FOO
sed -i -e 's/^M/FOO/g' input

OR easy to use sed syntax to remove carriage return in Unix or Linux:

sed 's/\r$//' input > output
sed 's/\r$//g' input > output 
# GNU/sed syntax
sed -i 's/\r$//g' input

To replace a carriage return (CR) with sed command

The syntax is:
sed 's/\r/YOUR-replacement-TEXT-HERE/' input > output
sed 's/\r/YOUR-replacement-TEXT-HERE/g' input > output
sed 's/\r/foo/g' input > output

How to verify ^M in a text file

Use the cat command as follows:
cat -v input
Sample outputs:

Fig.01: cat and sed command in action to delete carriage returns and linefeeds (CRLF)
Fig.01: cat and sed command in action to delete carriage returns and linefeeds (CRLF)


A note about deleting or replacing a linefeed (LF) with sed on Unix or Linux

Use the following syntax if you do not want to delete \n (new line):
sed -i ':a;N;$!ba;s/\n//g' input
OR
sed ':a;N;$!ba;s/\n//g' input > output
See sed command man page for more info.

Remove a carriage return with dos2unix command

You can also use dos2unix command to converts text files from the DOS format to the Unix format:

dos2unix input
dos2unix -b input

The tr command syntax

To delete a CRLF:
tr -d '\r' < input > output

Posted by: Vivek Gite

The author is the creator of nixCraft and a seasoned sysadmin, DevOps engineer, and a trainer for the Linux operating system/Unix shell scripting. Get the latest tutorials on SysAdmin, Linux/Unix and open source topics via RSS/XML feed or weekly email newsletter.



 24 comment

  1. On my version of sed, GNU sed 4.2.1, the above command sets perform quite differently:

    sed ‘/^M/d’ input removes the lines with ^M in them, whereas
    sed ‘s/^M//’ input removes just the ^M from the lines.

    The latter was the desired behaviour in my case, I hope this helps.

  3. In some cases I had to check that all files have no carriage return at the end of the line.

    With the command “be” this task although it seems somewhat labored is greatly facilitated.

    I leave here the statement that I use for this purpose:

    #tested with GNU sed
sed -e '/^M/d' -i -r $(find . -type f)

  7. I was saying: tr -d ‘\r’ \ output; mv output input seems the comment for doesn’t escape \<.

  8. Hi
    Try this example: sed “s/\^M//g” testfile >testfile.out.
    It is important to put a Backslash to protect the ^M charactar!
    cheers and have a nice day ;-)

  9. For those of you saying this doesn’t work, I think I might have your answer. Please note this line in the article:

    “Type the following command (to get ^M type CTRL+V followed by CTRL+M):”

    You can’t just type the carat symbol and a capital M. You have to hit CTRL-V and then hit CTRL-M. I had quite a time figuring that out….but once you do it right, it works great.

    1. You’re right bogus,

      Personnally, I tried with copying-pasting the expression and it failed.
      We’ve got to type literally the keys sequence that you notice.

      Thanks !

  10. bogus, no, that didn’t work either. This really doesn’t work.
    I tried \{CTRL-V}{CTRL-M} inside my sed command’s string and that didn’t work
    I also tried {CTRL-V}{CTRL-M} without the \ and that didn’t work either.
    It does not find the carriage return on a Mac. You’re right that {CTRL-V}{CTRL-M} shows up as ^M and the cursor moves across both characters at once so it’s definitely treated as a CR instead of {Carat}{M} but sed on the Mac will not find it.

    Does anyone else have any ideas?

  11. Thanks for the tr method Valentin. It works and doesn’t require entering the actual CR code so it can be cut and pasted.

    tr -d ‘\r’

  15. Not every system has dos2unix installed on it.. and sometimes you are working on unix boxes and creating scripts because you administer an application on it, but are not the unix administrator yourself (and they won’t install it :D )

    it’s good to talk about ways to do things without 3rd party tools, just in case you don’t have them.

  17. I see it’s really old tutorial – but still on google is quite high in search results..

    The safets way to replace carraige sign is just by using ascii codes..

    sed -i 's|[\d13]||g' Input_file

    or

    sed 's|[\d13]||g' IN > OUT

  18. Osx uses freeBSD’s SED, which is not the Same as the GNU version of SED found on many Linux distros. The little differences can be really maddening. you can install the GNU version to make this syntax work. Google ‘install GNU SED on osx ‘

  19. There are two example of removing the CR:

    sed ‘s/\r//’ input > output

    sed ‘s/\r$//’ input > output

    What does the $ signify in the 2nd example?

    1. Thank you.
      Does that mean that the first one will remove the CR from anywhere in the line and the 2nd will only remove it from end-of-line?

      I notice the comment above the 2nd example said:
      “OR easy to use sed syntax to remove carriage return in Unix or Linux:”
      And the only difference was the “$” inclusion. Is that significant?

    Have a question? Post it on our forum!



Tagged as: Tags, , , , , , , , , , , , , , 

来自  https://www.cyberciti.biz/faq/sed-remove-m-and-line-feeds-under-unix-linux-bsd-appleosx/


vim 打开文件末尾带有^M的解决办法

2016年02月23日 11:57:10  阅读数：11757 标签： shell脚本linux换行符元宵节 更多
个人分类： 繁杂的小问题


版权声明：如发现任何问题，欢迎留言讨论；如有错误，请及时指出，确认后会及时修改。谢谢！ https://blog.csdn.net/laoding1993/article/details/50722029

一、背景 
昨天写了一个expect脚本用来做 板卡的自动化测试，获取framer设备的link up/down信息 定向到log文件里面。为了方便分析和处理，使用shell 脚本处理这个log文本。 
主要思路是将log中的特定行输出至新的new_log文件，对new_log文件中的词语进行拆分、比较，对设备link up/down 的状态信息进行统计。 
初步写了一个shell脚本后，发现对设备的统计信息明显不对。为了测试程序，伪造了一个log文本，就是自己复制粘贴的数据；处理之后，发现统计信息没什么问题。。继续对源文件进行处理，还是有问题。百思不得其解。。。。 
二：发现问题 
经过伪造的log与new_log的对比分析发现，new_log文本里每行的末尾多了一个^M;利用shell鞋的分词程序发现，这个^M竟然与末尾单词是一个整体，怪不得统计信息有问题；单纯比较字符串的话，肯定是有问题的； 
vim打开 new_log 的显示如下：

     CPU Normal^M
     Framer Host Link Up^M
     CPU Normal^M
     Framer Host Link Down^M


分词log如下：

CPU 
Normal^M 
Framer 
Host 
Link 
Up^M

三、解决问题 
发现这个问题之后，就得想办法去除^M 以及找到^M出现的原因； 
经过百度，发现：

1. 在windows下的文本文件的每一行结尾，都有一个回车('\n')和换行('\r')
2. 在linux下的文本文件的每一行结尾，只有一个回车('\n');
3. 在Mac下的文本文件的每一行结尾，只有一个换行('\r');


而在linux下打开windows编辑过的文件，就会在行末尾显示^M; 
这个^M的在vim下的输入是：

ctrl+v <----> ^
ctrl+m <----> M


四：解决方法 
1：首先想到的字符串的替换 
利用VIM的命令行输入：

%s/^M$//g


解释：% 指匹配整个文件，s 是置换的意思，^M 注意要用 Ctrl + V Ctrl + M 来输入，M 后面的 $ 代表匹配行尾的内容，最后的 g 则表示每行中匹配到的内容都要置换； 
2：dos2unix工具

dos2unix Filename


3：批量转换

find ./ -type f -print0 | xargs -0 dos2unix


或者

find ./ -type f print0 | xargs -0 sed -i 's/^M$//'


即可。 
五：后记 
找到了解决方法，问题自然很快就解决啦。目前自动化脚本运行非常良好。分析结果，很直观。不过，这只是查找问题的第一步。 
BTW，昨天是元宵节，解决完这个问题就直接回去了，今天补充的昨天的记录。



来自  https://blog.csdn.net/laoding1993/article/details/50722029



转:在linux中vi 文件里行尾奇怪的^M及解决

Linux编辑器vim中删除行尾的^M

　　有时候，在 Linux 中使用打开曾在 Windows 中编辑过的文件时，会在行尾看到 ^M 字符。看起来总是感觉很别扭。

　　删除方法如下：

　　在 Vim 的命令模式中输入 :%s/^M$//g 后，回车即会自动删除该文件中的所有 ^M 字符。

　　注意： ^M 要用 Ctrl + v， Ctrl + m 来输入， 用键盘直接敲^和M是不行的! 后面的 $ 代表匹配行尾的内容，最后的 g 则表示每行中匹配到的内容都要置换--全局替换，否则只替换每行中匹配到的第一个。

　　又脚本删除方法：

　　cat file | col -b > file.1 // 这个可以去掉，但是生成文件里汉字变乱码

　　sed -e 's/.$//g' file // 正确

　　sed ‘s/^M//' file // 正确，但是 ^M = Ctrl + v, Ctrl + m

　　附录：^M 另一个会出现的情况是文件在 windows 和 Linux 系统间通过 Ftp 传送。这是建议方法是：强行设定 ftp 方式为 ascii 方式就可以了啊! 当然也可以通过软件 dos2unix 搞定。// 我没尝试过哈

炊烟起了；夕阳下了；细雨来了 read and connect 匍匐前进,


来自  https://www.cnblogs.com/feiyun8616/p/6363054.html 

普通分类: 
linux
Powered by Drupal

友情链接

校园好文网

 www.shipingzhong.cn  个人技术网_前端_后台    备案号:苏ICP备18010659号-2