Questions tagged [awk]

AWK is an interpreted programming language designed for text processing and typically used as a data extraction and reporting tool. AWK is used largely with Unix systems.

-1
votes
2answers
18 views

awk - file manipulation where a pattern need to be removed

I have a file with following records: www.test.com www2.test.com www.foo.com www3.foo.com www.example.com I need to remove 'www.' from each record where its present in the file using one-liner awk ...
0
votes
1answer
46 views

Command “echo” has no effect in awk

I have a line of code that I need to run in a linux terminal and it`s not going very well. What i`m doing is trying to output some variables obtained from my postfix mail queue to a file. For now I ...
-3
votes
0answers
30 views

Can somebody help me to convert AWK command to Powershell? [on hold]

Can somebody help me to convert AWK command to Powershell ? awk -F\, 'NR==FNR{a[$1,$2]=$3OFS$4;next}{$4=a[$1,$2];print}' OFS='\,' site.csv app.csv |awk -F\, {'print $1OFS$2OFS$3OFS$4'} OFS='\,' > ...
0
votes
1answer
17 views

Is there a line overflow in awk? [duplicate]

I want to transform a file via awk from: tid|uid|pid|date|type|price|quantity 12345678900000tid1|123uid1|12pid1|2019-01-01|order|1.97|10 to: time|userID|transactioID|group|itemID|transactionType|...
0
votes
3answers
47 views

Get the line number of the first line matching second pattern

Is it possible using awk or sed to get the line number of a line such that it is the first line matching a regex after another line matching another regex? In other words: Find line l1 matching ...
-1
votes
0answers
28 views

awk, calculation of the lowest value

I would like to print my solution in awk but the results is e.g. 1.28392E-27 and awk probably doesnt see it. I used "%.6e\n", or "%.6g\n", or "%16.13E \n". None of them showing xxE-27 but only 0.000E+...
0
votes
1answer
18 views

getting fatal: attempt to use a scalar value as array error while creating join between two files

I am creating a full outer join between two files and to do so, I am using awk to store both the files records in separate array and then print them later by using two for loops. awk 'FNR==NR{a[NR]=$...
1
vote
2answers
35 views

How to insert argument in awk script?

I'm writing a shell script which shut down some services and trying to get its pid by using the following awk script. However, this awk script can't get pid. What's wrong with that? ps -ef | awk -v ...
1
vote
1answer
20 views

Extract lines where a character match in 3rd column [duplicate]

I need to extract the lines where my 3rd column is equal to 38.I am trying the following command : awk '{printf "$3=38"}' file TRINITY_DN66807_c0_g1_i1 4 * 0 0 * * 0 0 ...
0
votes
1answer
30 views

How to join column with several hits with Awk

I have a tabulate file like this, joj001.txt: C00299 map01 C00125 map65 C00299 map13 and a csv file , dora.csv: V1 V2 V3 D12 C00299 4 E10 C01832 5 I want to add a column (or ...
2
votes
2answers
55 views

Are these awk commands vulnerable to code injection?

I was unsure on how to correctly script a particular awk command which uses a shell variable, when I read the answers to How do I use shell variables in an awk script?. The accepted answer ...
0
votes
1answer
18 views

Passing variables to awk script with just a couple of awk commands

I am very new to awk so apologies for this question. I have made a script with lots of awk commands (called it "myscript.awk", e.g: awk -v f=5 -v t=62 '{ for (i=f; i<=t;i++) printf("%s%s", $i,(i==...
0
votes
1answer
45 views

How can one sed the output of a variable above a certain line? [on hold]

I need to modify a file above the line containing this string: [ moleculetype ] The insert I am trying to do is saved in a variable like this one: ATOMTYPES=$(sed -n '/^\[\ atomtypes\ \]/,/^\[\ ...
0
votes
2answers
49 views

Remove row from a file perl

I have file with | delimited row in that i want to add check on the value of 8th position if the value matches i want to remove that row from the file and if it not matching i want to leave that in ...
3
votes
2answers
84 views

Remove the first capitalized word after a period

I would like to be able to remove the first word written in capital after the period. The goal is to remove the first word written in capital letters even if the sentences on the same line are two. In ...
1
vote
3answers
57 views

How to merge column output to the end of a row in the previous column?

I have a .csv file containing three columns and I need to merge the value of column 2 with the end of the row of column 1. The .csv file contains thousands of rows and this needs to be done for each ...
0
votes
2answers
41 views

How can I replace all instances of a character between the Nth and Kth instance of that character?

I am working with a large number of CSV files, and in one of the columns, the field itself contains commas. Unfortunately, this column hasn't been enclosed in quotes, so it's causing an issue with ...
0
votes
1answer
27 views

output i get is “ snmp-server location Nova South, United Kingdom (GHTY2916)” , I want to get all the words after “location”

if i run the command in cisco ios device " show run | i location " , i may get the output as "snmp-server location Nova South, United Kingdom (GHDY2892)" I want to fetch all the words after 'location'...
-4
votes
1answer
53 views

Remove only character between two known strings

I want to remove the % symbol when it occurs between // and /, but not outside of this. Here is an example: https://delete%me.com/butnot%this https://donotdelete.me/test I would use sed, grep or ...
1
vote
3answers
38 views

How can you compare entries between two columns in linux?

I am trying to figure out whether the first letter of an amino acid is the same as its letter code. For example, Glycine begins with G and its letter code is also (G) On the other hand, Arginine ...
0
votes
1answer
47 views

How to use index front of $ in a loop?

I am struggling with a simple loop. I want to get: awk '{print $3}' z.csv > col1.csv awk '{print $4}' z.csv > col2.csv ... awk '{print $(i+2)}' z.csv> col(i).csv Here what I tried so far: ...
0
votes
0answers
30 views

Script to write data in other file [duplicate]

I just discovered that my teacher wants to make everything in awk and I don't know nothing, so that's why I here; I have a file with marks of all students, main_list: #name surname ...
1
vote
2answers
20 views

Why some GNU parallel input include quotes?

Gnu parallel input (e.g. from a pipe) automatically single quotes input that contains space or symbols like / and : Is there a reason for that? How can I print the input as it is without any quote? ...
0
votes
1answer
24 views

how to optimize this awk script?

I browse 2 files with awk. I browse the first file and store the columns I need in arrays. I use after these arrays to make a comparison with a column (8) of the second file. my script runs very ...
1
vote
2answers
27 views

Calculate average and write it in other file [duplicate]

I have a list of students with ID and marks, and I need to make another one with their average marks. main_list: #name surname student_index_number course_group_id lecturer_id list_of_marks athos ...
1
vote
1answer
22 views

How to replace the values of a file by the values of another file?

I explain my problem : I have two files , one which looks like this (it is a .tsv one, each row has not necessarly the same number of columns) : OTU0001 Archaea OTU0002 Archaea;Aenigmarchaeota;Deep ...
0
votes
1answer
39 views

Unable to extract fastboot command with awk

I am writing a short script to collect the Android Phone information with fastboot. With following command fastboot getvar product I can get return value like this product: "name" ...
-3
votes
0answers
22 views

AWK help to cut some value from a line

I want to print/cut string from file which contains something like below I want to support CH: UTF-8 fully in my web application I want to support CH: UTF-8 fully in my web application I want to ...
1
vote
3answers
44 views

How to extract patterns from a file and fill an bash array with them?

My intent is to write a shell script to extract a pattern ,using regular expressions, from a file and fill an array with all the ocurrences of the pattern in order to foreach it. What is the best way ...
1
vote
3answers
31 views

Extracting value from ipmitool text file using bash

I'm using Linux (Raspberry Pi) and bash. I'm using the Pi to pull IPMI sensor info from a Dell iDRAC. I've managed to pulled the sensor information off to a txt file, sample below: Fan1 RPM | ...
2
votes
0answers
43 views

connecting to a Unix domain socket from an AWK script

I want to open a Unix domain socket for both reading and writing from an AWK script. I'm using Gnu AWK. Accoring to this guide, ordinary sockets could be opened as "/net-type/protocol/local-port/...
0
votes
1answer
35 views

Populate a value in a particular column in csv

I have a folder where there are 50 excel sheets in CSV format. I have to populate a particular value say "XYZ" in the column I of all the sheets in that folder. I am new to unix and have looked for ...
0
votes
1answer
22 views

sed replace text in-between two strings

I want to replace the host id in number of files and the host id is different in each file, so I have to replace the content between the tags <hostid> and </hostid>, each file has only one ...
0
votes
5answers
54 views

Strip last field

My script will be receiving various lengths of input and I want to strip the last field separated by a "/". An example of the input I will be dealing with is. this/that/and/more But the issue I am ...
-1
votes
1answer
42 views

Passing multiple variables from a Bash Script to an Expect Script

I've been trying to get an expect/bash script that can read each line of a CSV file and pull both the hostname address and the password; as these are all different for each MikroTik I am trying to ...
2
votes
3answers
63 views

Average marks from list

Sorry if I don't write good, it's my first post. I have a list in one file with the name, id, marks etc of students (see below): And I want to calculate the average mark in another file, but I don't ...
1
vote
3answers
43 views

How to extract a string between two characters?

I have a string "foo [string] bar" and want to extract "string" out of it, dismissing all content before and after. My grep has not the -P option, so I tried with sed. echo "foo [string] bar" | sed -...
-10
votes
0answers
44 views

unable to convert paragraphs to columns using shell script [on hold]

i have the input file like this : input and the output i am expecting is like below output Could you please help me out here.
-3
votes
2answers
26 views

awk if else statement syntax error. What is causing it?

There's a syntax error in awk if else statement which I got the code from another question and unable to fix it. Bash one-liner code to output unique values. Can someone correct the statement. awk ...
0
votes
3answers
25 views

Add string to columns in bash

I have a comma-delimited file to which I want to append a string in specific columns. I am trying to do something like this, but couldn't do it until now. re1,1,a1e,a2e,AGT re2,2,a1w,a2w,AGT re3,3,...
-1
votes
4answers
57 views

Concatenate the sequence to the ID in fasta file

Here is my input file >OTU1;size=4; ATTCCGGGTTTACT ATTCCTTTTATCGA ATC >OTU2;size=10; CGGATCTAGGCGAT ACT >OTU3;size=5; ATTCCCGGGATCTA ACTTTTC The expected output file is: >OTU1;size=4;...
0
votes
0answers
15 views

Defining a reference to the eBird data file, problem with AWK connection in r

I'm using r to extract subsets of the Basic Dataset via AWK, and having problems defining a reference to the eBird data file. To troubleshoot this I'm running r as an administrator whilst following ...
1
vote
4answers
45 views

How to filter the OTU by counts with AWK?

I am trying to filter all the singleton from a fasta file. Here is my input file: >OTU1;size=3; ATTCCCCGGGGGGG >OTU2;size=1; ATCCGGGACTGATC >OTU3;size=5; GAACTATCGGGTAA >OTU4;...
2
votes
3answers
30 views

awk not printing output to secondary file, syntax error?

More with my radio logs... So far the work I have completed is great, push notifications are going out perfectly, too well in fact. I am getting bombarded with every accidental connection. I want to ...
2
votes
2answers
44 views

Finding max value using AWK

I have a file with two column data and I want to find the max value and print it. file =load_measure 11:20,18.03 11:25,17.85 11:30,18.24 11:35,19.19 11:40,18.45 11:45,17.53 11:50,17.56 11:55,17.60 12:...
1
vote
3answers
41 views

What do I use to manipulate text files and extract data out of it and store the output in a tabular format?

I have a text file which has daily data for various stocks identified by PERMNO. So the text file looks like this: PERMNO = 1234 PERMNO = 2134 Market data: Date | Price | Return | Volume --...
0
votes
1answer
48 views

No output from ''tail -f | gawk | tee''; either ''tail -f | gawk'' or ''gawk | tee'' works fine

tee works fine if placed before the gawk command, but not at all if placed after. Is it not possible to use after gawk? #!/bin/bash CURRENT_DATE=`date -u +%Y-%m-%d` tail -fn0 /var/log/pi-star/MMDVM-"...
2
votes
3answers
34 views

Transpose rows to columns keeping some rows as reference in shell

I have a file that I need to transpose, it looks like this: Person ID| commute to work in hours?| Happiness score?| work satisfaction score? 1234| 2 | 8 | 7 5678| 1 | 7 | 6 6789| 0.5 | 9 | 6 I need ...
1
vote
1answer
69 views

Make the syntax of the command line parameter match the name of a folder if it looks like it

I am quite new to bash scripting so i don't know how to proceed with sed or awk or any other bash scripting tool. My issue is: I am supposed to run a sh script which take one parameter. Due to some ...
0
votes
1answer
38 views

Assistance with transpose function using Bash

I want to transpose a given matrix provided in a grading script. I cannot use AWK. The function must be written in Bash. I have the following code where the AWK works perfectly with the provided ...