Posts

Showing posts from November, 2016

Use your browser as Notepad

Image
We all uses notepad or sticky note or some editor to take immediate notes on something like code snippet, ideas, blog content, todo. Recently I came to know that we can use our browser just like notepad. The trick hacks around Data URI scheme and html contenteditable attribute. All you need to do is type the following code into the browser's URL bar: data : text / html , < html contenteditable > It will make your page as editable just like notepad. If you want to save your content, do the usual browser save( CMD+S  for OSX). It will save your content as html file. You can also bookmark above data url to make it easier. Editor with little bit styles by bichiliad data : text / html , < html contenteditable >< style > body { color : # 333 ; width : 960 px ; margin : 0 auto ; display : block ; height : 100 % ; font - size : 36 px ; padding : 20 px ;} < /style...

JAWS: THE JAVASCRIPT + AWS STACK.

Image
JAWS is a stack from Amazon web services(AWS) to ease the development of massive scalable web applications. It is trying to solve important problems in scalable web application development. 1. No Backend servers : All web and mobile application needs backend server and database server. Since the JAWS back-end is comprised entirely of AWS Lambda Functions, you don't need to write your back end server in Node, Ruby, PHP or python. A back-end comprised of Lambda functions comes with a ton of concurrency and you can easily enable multi-region redundancy. So there is no need for scaling/deploying/maintaing/monitoring servers again. 2. Cheap : Lambda functions run only when they are called, and you only pay for when they are run. You can build your app using following AWS services Lambda  - Build worker tasks that you can spawn and scale infinitely. DynamoDB  - Managed, NOSQL data storage API Gateway  - Launch an API with...

Sending Logstash Logs to Loggly

Image
We are using  Logstash  for our log management. We integrated logstash with  ElasticSearch , and  Kibana UI . However it does not work very well to us. So we were looking for possible tools to visualize the logs without much work. Finally, we decided to use  Loggly . Then we just follow the steps provided by loggly in their  documentation . Unfortunately it does not work for Logstash version 1.5. After spending few hours, we made it work. Here are the steps. Install Loggly Plugin Since Logstash version 1.5, there is a new plugin management system. To install a Logstash Loggly plugin, use the following command: sudo bin / plugin install logstash - output - loggly Create Logstash configuration Create a  logstash-loggly.conf  file with following contents output { loggly { key => "TOKEN" host => "logs-01.loggly.com" proto => "https" } } ...

Restrict user access to Single S3 Bucket using Amazon IAM

Image
S3 becomes de-facto standard for publishing files in the internet. When you work with the team, you might want to restrict an access of single S3 bucket to specific users. You can do it in  IAM  as follows Create  Group Create  User . Then use  Manage Password  to add a password for your user. Add User to the created Group Create and Attach Permission Policy for the group. Step 1, 2 and 3 are straight forward. But creating and attaching permission policy needs some attention. In  Group  tab, go to created group. Then click the  Permissions  tab and Click the  Attach Policy  button. You’ll be taken to  Set Permissions  page where you can Manage User Permissions. Here you can select a  Select Policy Template  option, then find the  Amazon S3 Full Access  and click  Select  button. You will be prompted with  Policy Name  and...

Encrypting Ephemeral Storage and EBS Volumes on Amazon EC2

Image
Over the years, Amazon has repeatedly recommended that customers who care about the security of their data should consider encrypting information stored on disks, whether ephemeral storage ( /mnt ) or EBS volumes. This, even though they take pains to ensure that disk blocks are wiped between uses by different customers, and they implement policies which restrict access to disks even by their own employees. There are a few levels where encryption can take place: File level. This includes tools like GnuPG, freely available on Ubuntu in the  gnupg  package. If you use this approach, make sure that you don’t store the unencrypted information on the disk before encrypting it. File system level. This includes useful packages like  encfs  which transparently encrypt files before saving to disk, presenting the unencrypted contents in a virtual file system. This can even be used on top of an  s3fs  file system letting you store encrypted data on S3 with ea...

Improving Security on EC2 With AWS Identity and Access Management (IAM)

Image
Amazon launched a public preview of  AWS Identity and Access Management  (IAM) which is a powerful feature if you have a number of developers who need to access and to manage resources for an AWS account. A unique IAM user can be created for each developer and specific permissions can be doled out as needed. You can also create IAM users for system functions, dramatically increasing the security of your AWS account in the event a server is compromised. That benefit is the focus of this article using an example frequently cited by EC2 users: Automating EBS snapshots on a local EC2 instance without putting the keys to your AWS kingdom on the file system. Before the release of AWS IAM, if you wanted to create EBS snapshots in a local cron job on an EC2 instance, you needed to put the master AWS credentials in the file system on that instance. If those AWS credentials were compromised, the attacker could perform all sorts of havoc with resources in your AWS account and charg...

Automatic Termination of Temporary Instances on Amazon EC2

Image
I frequently fire up a temporary Ubuntu server on Amazon EC2 to test out some package feature, installation process, or other capability where I’m willing to pay a few pennies for a clean install and spare CPU. I occasionally forget that I started an instance and leave it running for longer than I intended, turning my decision to spend ten cents into a cost of dollars. In one case, I ended up paying several hundred dollars for a super-sized instance I forgot I had running. Yes, ouch. Because of this pain, I have a habit now of pre-scheduling the termination of my temporary instances immediately after creating them. I used to do this on the instance itself with a command like: echo "sudo halt" | at now + 55 min However, this only terminates the instance if its root disk is instance-store (S3 based AMI). I generally run EBS boot instances now, and a shutdown or halt only “stops” an EBS boot instance by default which leaves you paying for the EBS boot volume at, say, $1...

Uploading Personal ssh Keys to Amazon EC2

Image
Amazon recently launched the ability to upload your own ssh public key to EC2 so that it can be passed to new instances when they are launched. Prior to this you always had to use an ssh keypair that was generated by Amazon. The benefits of using your own ssh key include: Amazon never sees the private part of the ssh key (though they promise they do not save a copy after you downloaded it and we all trust them with this) The private part of the ssh key is never transmitted over the network (though it always goes over an encrypted connection and we mostly trust this) You can now upload the same public ssh key to all EC2 regions, so you no longer have to keep track of a separate ssh key for each region. You can use your default personal ssh key with brand new EC2 instances, so you no longer have to remember to specify options like  -i EC2KEYPAIR  in every  ssh ,  scp ,  rsync  command. If you haven’t yet created an ssh key for your local s...

EBS Boot Instance Stop+Start Begins a New Hour of Charges on EC2

Image
I was wondering if stopping and starting an EBS boot instance on EC2 would begin a new hour’s worth of charges or if AWS would not increase your costs if the stop/start were done a few minutes apart in the same hour. For some reason, I had assumed that it would start a new hour of fees, possibly because of my experience with the somewhat unrelated terminating old instances and starting new instances. However, we decided it would be easy to test, so here are the results. I tested with an Ubuntu 10.10 Maverick 32-bit server EBS boot AMI on the  m1.small  instance type in the  ap-southeast-1  (Singapore) region. The AMI should have no effect on charges, so these results should apply to any OS you run on EC2. I used an AWS account that did not have any EC2 instance fees in the Singapore region that month (Scott’s idea) so that this activity would be easy to see as the only charges on that account. $ ec2-run-instances --region ap-southeast-1 --key KEYPAIR ami-6...

Boot EC2 Instance With ssh on Port 80

Image
In a  thread on the EC2 forum , Marko describes a situation where an outbound firewall prevents the ability to ssh to port 22, which is the default port on all EC2 instances. In that thread, Shlomo Swidler proposes creating a  user-data script  that changes sshd to listen on a port the firewall permits. Here’s a simple example of a user-data script that does just that. Most outbound firewalls allow traffic to port 80 (web/HTTP), so I use it in this example. The first step is to create a file containing the user-data script: cat <<'EOM' >user-data-ssh-port-80.txt #!/bin/bash -ex perl -pi -e 's/^#?Port 22$/Port 80/' /etc/ssh/sshd_config service sshd restart || service ssh restart EOM The first statement changes the sshd config to listen on port 80 instead of port 22, and the second statement restarts sshd so it will start using this new configuration. Now you can run a new instance on Amazon EC2, passing in this user-data ...