Академический Документы
Профессиональный Документы
Культура Документы
July 2, 2014
Filed under: Linux Related Tags: lusca automated script, lusca install script Syed Jahanzaib / Pinochio~:) @ 12:22 PM
i
13 Votes
Following is an automated script to install for with aggressive content caching support including some video web sites
like YOUTUBE and few others as described in my other article @
h2ps://aacable.wordpress.com/2014/04/21/howto-cache-youtube-with-squid-lusca-and-bypass-cached-videos-from-mikrotik-queue/
I will add more n more functions as soon as I get some free time, like congurable options via choice menu like cache size, mem, and other
variables.
Update Ubuntu
Install some components required for Compilation of Lusca/Squid package
Backup squid.conf if already in /etc/squid.conf with squid.conf.old, stop any running squid instance
Download LUSCA r14942 source package to /temp folder and compile it
Download squid.conf and storeurl.pl from the internet and place them in /etc/squid.conf
Create cache directory like in /cache-1 and default cache size is 5 GB
add squid in /etc/rc.local so it may start auto upon system reboot
You should modify all options in after installation , like cache_dir, cache_mem and others as per your network and hardware
specications.
mkdir /temp
cd /temp
touch lusca_install.sh
chmod +x lusca_install.sh
nano lusca_install.sh
1 #!/bin/bash
2 # Version 1.0 / 2nd July, 2014
3 # LUSCA r14942 Automated Installation Script for Ubuntu flavor / jz
4 # Syed Jahanzaib / aacable @ hotmail.com / https://aacable.wordpress.com
5
6 # Setting Variables . . . [JZ]
7 # You can change the URL if default url is not accessible in some cases.
8 #URL=http://aacable.rdo.pt/files/linux_related/lusca
9 URL=http://wifismartzone.com/files/linux_related/lusca
10 SQUID_DIR="/etc/squid"
11 CACHE_DIR="/cache-1"
12 pid=`pidof squid`
13 osver=`cat /etc/issue |awk '{print $1}'`
14 squidlabel="LUSCA_HEAD-r14942"
15
16 # Colors Config . . . [[ JZ . . . ]]
17 ESC_SEQ="\x1b["
18 COL_RESET=$ESC_SEQ"39;49;00m"
19 COL_RED=$ESC_SEQ"31;01m"
20 COL_GREEN=$ESC_SEQ"32;01m"
21
22 # OS checkup for UBUNTU
23 echo -e "$COL_GREEN Lusca r14942 Automated Installation Script ver 1.0 for Ubuntu . . .$COL_RESET"
24 echo -e "$COL_GREEN Checking OS version, as it must be Ubuntu in order to Continue . . .$COL_RESET"
25 if [[ $osver == Ubuntu ]]; then
26 echo
27 echo -e "$COL_GREEN Ubuntu is installed with following information fetched. $COL_RESET"
28 lsb_release -a
29 sleep 3
30 else
31 echo -e "$COL_RED Sorry, it seems your Linux Distribution is not UBUNTU . Exiting ...$COL_RESET"
32 exit 1
33 fi
34
35 # Make sure only root can run our script / Checking if user is root, otherwise exit with error [[Jz]]
36 echo
37 echo -e "$COL_GREEN Verifying if you are logged in with root privileges . . .$COL_RESET" 1>&2
38 FILE="/tmp/out.$$"
39 GREP="/bin/grep"
40 if [ "$(id -u)" != "0" ]; then
41 echo
42 echo -e "$COL_RED This script must be run as root, switch to root now . . .$COL_RESET" 1>&2
43 exit 1
44 fi
45
46 # Clearing previous download if any in /tmp folder
47 echo
48 echo -e "$COL_GREEN Clearing previous downloads if any in /tmp folder to avoid duplication$COL_RESET"
49 sleep 3
50
51 rm -fr /tmp/squid.conf
52 rm -fr /tmp/storeurl.txt
53 rm -fr /tmp/storeurl.pl
54 rm -fr /tmp/LUSCA_HEAD-r14942*
55
56 # Checking IF $URL is accessible m if YES then continue further , otherwise EXIT the script with ERROR ! [[ JZ .. .
57 echo
58 echo -e "$COL_GREEN Checking if $URL is accessible in order to proceed further. . .!! $COL_RESET"
59 cd /tmp
60 wget -q $URL/squid.conf
61 {
62 if [ ! -f /tmp/squid.conf ]; then
63 echo
64 echo -e "$COL_RED ERROR: Unable to contact $URL, or possibly internet is not working or your IP is in black list at
64 echo -e "$COL_RED ERROR: Unable to contact $URL, or possibly internet is not working or your IP is in black list at
65 echo -e "$COL_RED ERROR: Please check manual if $URL is accessible or not or if it have required files, JZ !! $COL_
66 exit 0
67 fi
68 }
69 rm -fr /tmp/squid.conf
70 sleep 6
71 # Moving further . . .
72
73 clear
74 echo -e "$COL_GREEN You are logged in with root ID, Ok to proceed further . . .!! $COL_RESET"
75 echo
76
77 ################################################################## [zaib]
78 echo
79 echo -e "$COL_GREEN Updating Ubuntu first . . . !! $COL_RESET"
80 apt-get update
81 echo
82 echo
83 echo -e "$COL_GREEN Installing required components . . . !! $COL_RESET"
84 sleep 3
85 apt-get install -y gcc build-essential libstdc++6 unzip bzip2 sharutils ccze libzip-dev automake1.9 l
86
87 # Clearing OLD data files . . .
88 {
89 if [ -f $SQUID_DIR/squid.conf ]; then
90 echo
91 echo
92 echo -e "$COL_RED Previous SQUID configuration file found in $SQUID_DIR ! renaming it for backup purpose . . . $COL_
93 mv $SQUID_DIR/squid.conf $SQUID_DIR/squid.conf.old
94 else
95 echo
96 echo
97 echo -e "$COL_GREEN No Previous Squid configuration have been found in $SQUID_DIR. Proceeding further $COL_RESET"
98 fi
99 }
100
101 # Checking SQUID status if its already running - check by PID
102 if [ "$pid" == "" ]; then
103 echo
104 echo
105 echo -e "$COL_GREEN No SQUID instance found in memory , so it seems we are good to GO !!! $COL_RESET"
106 else
107 echo
108 echo -e "$COL_RED SQUID is already running, probably you have some previous copy of SQUID installation, Better to st
109 echo
110 echo -e "$COL_RED KILLING PREVIOUS SQUID INSTANCE by killall -9 squid command !! $COL_RESET"
111 killall -9 squid
112 sleep 3
113 fi
114
115 # Downloading Squid source package [zaib]
116 echo
117 echo
118 echo -e "$COL_GREEN Downloading SQUID source package in /tmp folder. . . !! $COL_RESET"
119 sleep 3
120
121 # Checking if /tmp folder is previously present or not . . .
122 {
123 if [ ! -d "/tmp" ]; then
124 echo
125 echo
126 echo -e "$COL_RED /tmp folder not found, Creating it so all downloads will be placed here . . . $COL_RESET"
127 mkdir /tmp
128 else
129 echo
130 echo -e "$COL_GREEN /tmp folder is already present , so no need to create it, Proceeding further . . . $COL_RESET"
131 fi
132 }
133
134 cd /tmp
135
136 # Checking IF LUSCA_HEAD-r14942.tar.gz installation file have been ALREADY downloaded in /tmp to avoid duplication!
137 {
138 if [ -f /tmp/LUSCA_HEAD-r14942.tar.gz ]; then
139 rm -fr /tmp/LUSCA_HEAD-r14942.tar.gz
140 fi
141 }
142
143 wget -c http://wifismartzone.com/files/linux_related/lusca/LUSCA_HEAD-r14942.tar.gz
144
145 # Checking IF LUSCA_HEAD-r14942 installation file have been downloaded properly. if YEs continue further , otherwise
146 {
147 if [ ! -f /tmp/LUSCA_HEAD-r14942.tar.gz ]; then
148 echo
149 echo
150
151 echo -e "$COL_RED ERROR: SQUID source code package File could not be download or not found in /tmp/ !! $COL_RESET"
152 exit 0
153 fi
153 fi
154 }
155 echo
156 echo
157
158 echo -e "$COL_GREEN Extracting Squid from tar archive. . . !! $COL_RESET"
159 sleep 3
160 tar zxvf LUSCA_HEAD-r14942.tar.gz
161 cd LUSCA_HEAD-r14942/
162 mkdir /etc/squid
163
164 echo -e "$COL_GREEN Executing $squidlabel Compiler [jz] . . . !! $COL_RESET"
165 echo
166 cd /tmp/LUSCA_HEAD-r14942
167 ./configure --prefix=/usr --exec_prefix=/usr --bindir=/usr/sbin --sbindir=/usr/sbin --libexecdir=/usr/lib/squid --sy
168 echo
169 echo -e "$COL_GREEN Executing MAKE and MAKE INSTALL commands . . . !! $COL_RESET"
170 sleep 3
171 make
172 make install
173 echo
174 echo
175 echo -e "$COL_GREEN Creating SQUID LOGS folder and assiging permissions . . . !! $COL_RESET"
176 sleep 3
177
178 # Checking if log folder is previously present or not . . .
179 {
180 if [ -d "/var/log/squid" ]; then
181 echo
182 echo
183 echo -e "$COL_GREEN LOGS folder found. No need to create, proceeding Further . . . $COL_RESET"
184 else
185 echo
186 echo
187 echo -e "$COL_GREEN Creating LOG Folder in /var/log/squid and setting permissions accordingly (to user proxy) $COL_R
188 mkdir /var/log/squid
189 fi
190 }
191 chown proxy:proxy /var/log/squid
192 ## ** DOWNLOAD SQUID.CONF
193 echo
194 echo
195 echo -e "$COL_GREEN Downloading SQUID.CONF file from $URL and copy it to $SQUID_DIR. . . !! $COL_RESET"
196 sleep 3
197
198 # Checking IF SQUID.CONF File have been ALREADY downloaded in /tmp to avoid duplication! [[ JZ .. . .]]
199 {
200 if [ -f /tmp/squid.conf ]; then
201 rm -fr /tmp/squid.conf
202 fi
203 }
204
205 cd /tmp
206 wget $URL/squid.conf
207
208 # Checking IF SQUID.CONF file have been downloaded. if YEs continue further , otherwise EXIT the script with ERROR !
209 {
210 if [ ! -f /tmp/squid.conf ]; then
211 echo
212 echo
213 echo -e "$COL_RED ERROR: SQUID.CONF File could not be download or not found in /tmp/ !! $COL_RESET"
214 exit 0
215 fi
216 }
217 cp -fr squid.conf $SQUID_DIR
218
219 ## ** DOWNLOAD SQUID.CONF
220 echo
221 echo
222 echo -e "$COL_GREEN Downloading STOREURL.PL file from $URL and copy it to $SQUID_DIR. . . !! $COL_RESET"
223 sleep 3
224 cd /tmp
225
226 {
227 if [ -f /tmp/storeurl.txt ]; then
228 rm -fr /tmp/storeurl.txt
229 fi
230 }
231
232 wget $URL/storeurl.txt
233
234 {
235 if [ -f /tmp/storeurl.pl ]; then
236 rm -fr /tmp/storeurl.pl
237 fi
238 }
239
240 mv storeurl.txt storeurl.pl
241
242 # Checking IF STOREURL.PL file have been downloaded. if YEs continue further , otherwise EXIT the script with ERROR
242 # Checking IF STOREURL.PL file have been downloaded. if YEs continue further , otherwise EXIT the script with ERROR
243 {
244 if [ ! -f /tmp/storeurl.pl ]; then
245 echo
246 echo
247 echo -e "$COL_RED ERROR: STOREURL.PL File could not be download or not found in /tmp/ !! $COL_RESET"
248 exit 0
249 fi
250 }
251 cp -fr storeurl.pl $SQUID_DIR
252
253 echo
254 echo
255 echo -e "$COL_GREEN Setting EXECUTE permission for storeurl.pl . . . !! $COL_RESET"
256 chmod +x $SQUID_DIR/storeurl.pl
257
258 # Creating CACHE folders
259 echo
260 echo
261 echo -e "$COL_GREEN Creating CACHE directory in $CACHE_DIR , in this example,I used 5GB for cache for test ,Adjust i
262 sleep 3
263
264 # Checking if /cache-1 folder exist . . .
265 {
266 if [ ! -d "$CACHE_DIR" ]; then
267 echo
268 echo
269 echo -e "$COL_GREEN Creating cache folder in $CACHE_DIR , Default size is 5GB, you should set it accordingly to your
270 mkdir $CACHE_DIR
271 chown proxy:proxy $CACHE_DIR
272 chmod 777 -R $CACHE_DIR
273 squid -z
274 else
275 echo
276 echo -e "$COL_RED $CACHE_DIR folder already exists , Clearing it before proceeding. . . $COL_RESET"
277 rm -fr $CACHE_DIR/*
278 chown proxy:proxy $CACHE_DIR
279 echo -e "$COL_GREEN $CACHE_DIR Initializing Cache Directories as per the config . . . $COL_RESET"
280 echo
281 squid -z
282 chmod 777 -R $CACHE_DIR
283 fi
284 }
285
286 echo
287 echo
288 echo -e "$COL_GREEN Adding squid in /etc/rc.local for auto startup . . . !! $COL_RESET"
289 sed -i '/exit/d' /etc/rc.local
290 sed -i '/[/usr\/sbin\/squid]/d' /etc/rc.local
291 echo /usr/sbin/squid >> /etc/rc.local
292 echo exit 0 >> /etc/rc.local
293 echo
294 echo -e "$COL_GREEN Starting SQUID (and adding 10 seconds Pause for proper initialization). . . !! $COL_RESET"
295 squid
296 sleep 5
297
298 # Checking SQUID status via PID [zaib]
299 #if [ "$pid" == "" ]; then
300 #echo
301 #echo -e "$COL_RED ERROR: UNABLE to start SQUID, try to run with -d1N syntax and see where its showing error !! $COL
302 #else
303 ps aux |grep squid
304 echo
305 echo -e "$COL_GREEN $squidlabel is Running OK with PID number "$pid", no further action required, EXITING . . .$COL
306 echo
307 echo To view squid web access activity log, use command
308 echo -e "$COL_GREEN tail -f /var/log/squid/access.log $COL_RESET"
309 echo OR
310 echo -e "$COL_GREEN tail -f /var/log/squid/access.log |ccze $COL_RESET"
311 echo
312 echo -e "$COL_GREEN Regard's / Syed Jahanzaib . . . !! $COL_RESET"
313 echo
It will start installation and will show you the progress with all the action its doing [in colored rows, RED color shows error, Green Color shows
Ok/INFO].
if squid is successfully started , you can see its process via PS command
Comments (37)
37 Comments
Reply
2. Dear sir;;
How to block h2ps trac (facebook.com) by using squid with transparent proxy
Regards Assad
Reply
Reply
3. not working sir
Reply
4. nice share. This should be fresh ubuntu? Ill try later when I had to reinstall.
thanks
Reply
5. is this Lusca proxy working with youtube h2ps:// trac recently youtube changed their trac to h2ps://
Reply
no .
Reply
6. Thanks .sir ALLAH aap ko jaza e khair dey!
We Check this Script..
Reply
7. Sir.ubuntu 64 bit OS Architecture section & compiler for 64 bit
not mention in this article..ALLAH aap ko jaza e khair dey.
Reply
8. Dear Zaib thanks for your top POST
I want to cache youtube h2ps:// trac
if this possible with ssl?
Reply
Please read this.
h2ps://aacable.wordpress.com/2013/10/22/squid-3-and-ssl/
Reply
9. SirIs this Script also work on ubuntu 64bit & 64bit OS Architecture?
Because you dont write in the script ( 64 bit compiler of squid )
plzexplain ALLAH aap ko JAZA E KHAIR dey..
Reply
Yes this works with 32 bit as well as 64bit too.
Reply
10. Hi Syed, if i do LUSCHA with youtube for mikrotik. And want to do this script when computer restart. do i need to delete all the congs from
LUSCHA and youtube? Thank you
Reply
You dont need to do any thing further.
Reply
11. Hi Syed,
ERROR: Unable to contact h2p://wismarVone.com/les/linux_related/lusca, or possibly internet is not working or your IP is in black list at
destination server !!
ERROR: Please check manual if h2p://wismarVone.com/les/linux_related/lusca is accessible or not or if it have required les, JZ !!
Thanks
Comment by arnoldrodrigues August 1, 2014 @ 7:16 PM
Comment by arnoldrodrigues August 1, 2014 @ 7:16 PM
Reply
there is an alternate URL present there in the script. I will update the second URL link it tomorrow and will let u know
Reply
12. Hi, I have installed the same on Ubuntu 12.04 with hardware cong of I3-3.2Ghz with 8 GB RAM and 1TD HDD, in which 4GB allocated to
squid and 500GB Storage. The problem i face is after 1 day of working, it slows down or doesnt [the browsing part]. We need to restart the
lusca/squid in order to work again any solution for this. The Bandwidth is around 40mb in peak time and average 20-25mbps .
Reply
13. hello mr syed.. i have a problem when the squid startup it take about 2 to 3 mb for upload and download without redirect any website to
squidcan you help methanks you
Reply
hmm there is no such issue in default squid cong. You have to check at your router level that what data is being pulled/pushed from the
squid. probably someone is downloading OR probably Linux is updating something. You should wait few hours , maybe it will end itself.
Reply
14. Hi Syed. First of all, many thanks for your blog!! it has a really helpful for my job!! I would like to ask you about Lusca is not caching youtube
content anymore, i have tried on Ubuntu 12.04LTS and 14.04LTS with the same result: TCP_HIT: MISS, I have tried other tools/plugins as Squid
Video & Music Booster but nothing. Any piece of advice?
Reply
15. hi syed..its not working to caching youtube and facebook videos and applications..but working in other videos sites.. can i do anything to cache
videos from youtube and facebook videos?
Reply
16. can i add android application to be cached ?
Reply
17. Excellent article, congratulations for the work
which rules integration with mikrotik version 6.x with cache full?
Marti Souza
Brazil, Amazonas, Tee
Reply
18. thanks for your blogs sayd and please tell me about tips that cache youtube and facebook sites
cos
all users tell me that face book more slowly!!!
please tell me about that problem
Reply
19. youtube no caching because is h2ps. whats is the solution? thanks
Reply
20. Dear zaib please tell us how to cache youtube h2ps and another h2ps ?????
please help us
thank you
Reply
21. dear sir please help what is dscp this script ? for microtik router
Reply
22. i have problem : clientnatlookup nf getsockopt(so_original_dst) failed (92) protocol not available. how can solve?
Reply
23. Dear Sir, Please let me know that when installing ubuntu do we have to make partitions or it will make it automatically. if we do have to make
then please do let me know how and we can make and which partitions. thanks
Reply
24. Thanks sir it is working on latest ubuntu 14.04 LTS server, but how can I restart the lusca service without restarting the whole machine? Ive
tried sudo service lusca restart and sudo service squid restart but its not working or the service is not recognized.
Reply
25. Dear Sir, please tell me how to REMOVE ANY KIND OF PREVIOUSLY INSTALLED SQUID INSTALLATION COMPLETELY.
Thanks b4
dedmax
Reply
26. alhamdulillah! its working! but i need help
my network is like this :
ISP 1 (192.168.1.1) -|
MIKROTIK LAN Client (192.168.5.0/24) GW 192.168.5.254
ISP 2 (192.168.2.2) | |
|
Ubuntu server 12.04 LTS + LUSCA r14942 (192.168.4.2) GW 192.168.4.1
Reply
27. Hi!
when running squid proxy mikrotik graphs are stopped. not changed. please give me solution
Reply
28. Thanks .sir ALLAH aap ko jaza e khair dey!
We Check this Script nyc work but sir 1kia lusca may speedtest.ptcl.net ya speedtest.net cache nahi hoti aksar form per frnds ne speedtest ki
snaps di hoe hoti hain but mere pas speedtest cache nahi ho rahi
Reply
You shouldnt be caching speedtest.net like sites, they are dynamic and made to give you fresh result on every check. Why you want to get
FALSE results with it ?
Reply
29. what is the command to start the squid service in ubuntu 10.04??? and my all conguration is done but system shows unrecognized bash
29. what is the command to start the squid service in ubuntu 10.04??? and my all conguration is done but system shows unrecognized bash
command like this please tell what is actual command to start and stop the service of squid???
/etc/init.d in the path squid not here but all other are here.
Reply
30. Dear Mr.Zaib
I have installed the same on Ubuntu 12.04 with hardware cong of Xeon-2.8Ghz quad core with dual cpu with 20 GB RAM and 1.5TB HDD, in
which 4GB allocated to squid and 500GB Storage. The problem i face is after 1 or 2 day of working, it slows down or doesnt [the browsing part].
We need to restart the lusca/squid in order to work again any solution for this
Reply
RSS (Really Simple Syndication) feed for comments on this post. TrackBack URI (Uniform Resource Identier)
Blog at WordPress.com.