So-net無料ブログ作成

Programming language and framework in the era of JSON over HTTP [技術]

Nowadays, there are many programming projects that are bound by network transport and data types. HTTP(S) is probably the most popular transport and there are different HTTP server frameworks for different programming languages. So what would be the programming language of choice given a task to solve involving HTTP with a particular data type?

I am writing this post after completing a web service project that has been used for product and deployment testing for a fairly large system across multiple enterprises interconnected with each other. Here are some of the characteristics of the project:
  • HTTP is used as the transport for JSON data. There is no UI involved so no need for HTML or MVC.
  • Language choice is up to myself. No one cares what language I use.
  • Performance is not the primary goal.
  • Learning curve and ease of use are important when choosing a language and a framework.
Nowadays, the data structure is JSON. JSON is transported as text data, so an ideal programming language should be able to marshal/unmarshal (serialize/de-serialize) the data to and from native representation for easy retrieval and manipulation of data.

Another constraint comes from the fact that I am Japanese living in Japan: ideally the language can handle non-ASCII strings without much struggle.

I did not have enough time to try out all the different libraries and frameworks for my task, so I only looked at the documentation and examples before deciding on which language and framework to use.

That said, I am showing working examples of all the candidate languages below. I spent several nights for research and coding for completeness.

I have tested all below codes using curl like:
curl -X POST -H "Content-Type: application/json" -d '{"name":"Taro"}' http://localhost:3000/hello

Java
My language of choice had been Java for a long time mainly due to feature-rich IDEs. However, JSON support at the language level is not comparable to other languages like JavaScript or Python. At least that was my impression off the top of my head based on my past experience with the language. I did not want to use a library that forced the user to define data structures or classes up-front before starting the real task. In any case, org.json.JSONObject may be the choice if you are bound to Java and performance is not much of a concern. The interface is intuitive for a experienced Java developer. As for the REST framekwork, Spark appears simple enough. This is a working exmple that combines Spark with JSONObject:
package SparkSample;

import static spark.Spark.*;
import org.json.JSONObject;

public class MyApp {
    public static void main(String[] args) {
        post("/hello", (req, res) -> {
        	JSONObject body = new JSONObject(req.body());
        	return "Hello " + body.getString("name");
        });
    }
}
It is surprising that this amount of Java code can run a web server as I have not caught up with latest server side technology in Java. I do not even know which line of the above code starts the server that listens on a TCP port. Spark can make servlet a thing of the past.

However, I could not find these libraries while I was looking for a solution. Also, a Java project normally requires a build system like maven or gradle that forces me to write a verbose XML. In fact, I had trouble figuring our all the chores around gradle and Eclipse while trying to compile and run the above simple code. (This is the first time I used gradle.)

JavaScript
I have used this language for a few projects that ran entirely within web browsers, but I do not have real experience building a server app using JavaScript. I have found this framework called Express based on Node.js. Below is a working code based on the examples in the Express site:
const express = require('express')
const bodyParser = require('body-parser');
const app = express()
const port = 3000

app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

app.post('/hello', (req, res) => res.send('Hello ' + req.body.name))

app.listen(port, () => console.log(`Example app listening on port ${port}!`))
Clean but some unfamiliar syntax. As for JOSN handling, the body-parser will convert JSON in a POST body into native data structure in the 'req' object. Accessing properties is very easy.

Perl
I have been a reluctant user of this language from the beginning due to its excessive use of signs or special characters like $ % @ _ => -> ... I still think that these signs decrease readability and writeability of code. I occasionally use it for ad-hoc, small tasks, but not for this one.

Anyway, here is what I found: Mojolicious. JSON handling is already part of the library and data can be accessed very easily:
use Mojolicious::Lite;

post '/hello' => sub {
    my $c = shift;
    my $hash = $c->req->json;
    $c->render(text => 'Hello ' . $hash->{name}, format=>'txt');
};

app->start;
This is as intriguing as Spark, except, I still do not like the way $ is used in this language. I thought hash prefix is %. What is that $ in front of hash?

Python
I had not used Python much for serious projects. However, I realized that it could convert JSON data to and from native representation very easily. Syntax appeared familiar to me as it is closer to C++ and Java-style object-based programming than JavaScript or Ruby. Also there are HTTP/REST frameworks like Flask. Threading is there, too, and data structures that I would use internally were thread-safe. It appeared promising for what I was trying to achieve. Below is a working example:
from flask import Flask, request
app = Flask(__name__)

@app.route('/hello', methods=['POST'])
def hello():
    data = request.get_json()
    return "Hello " + data['name']

app.run(host='0.0.0.0', port=5000, debug=True)
Quite simple and the @ decorator is already familiar to me through Java annotations.

Flask already provides an API for accessing JSON data in a request coming from a client. 'request' is an object that is accessible from a Flask application, and the object has a method 'get_json()'. Then properties in a json object can be accessed in a straight forward way.

Ruby
I was once assigned a project that manipulated Excel spreadsheets by Ruby code. It was ok for that particular task, but I did not like the use of signs in this language either. A REST framework that I have found is Sinatra. I have arranged the example codes in their web pages as below:
require 'sinatra'

post '/hello' do
  request.body.rewind  # in case someone already read it
  data = JSON.parse request.body.read
  "Hello #{data['name']}!"
end
Looks clean, and it works. This is not as bad as I had imagined. It also shows how posted JSON can be handled.

Conclusion
I do not see a clear winner in this comparison. It is a matter of preference. Of course, above examples are only scratching the surface, and you will face idiosyncrasies in each language as you start utilizing more features.

I chose Python this time due to its syntax and because it does not require a build system like maven or gradle up-front.

On the other hand, to me, clear loser appears to be C++. (To be fair, I used to write tons of C++ codes at work!) I have been poking around for C++ libraries for this particular task, but without success. I want to hear from experts how simple it can get in C++ with Boost or some other libraries.
nice!(0)  コメント(0) 
共通テーマ:日記・雑感

TEDの動画から音声だけを取り出す方法 2 [英語]

シャドウイングのためにTEDの動画から音声だけを取り出す方法のふたつめです。とくにTEDの動画に限らず、YouTubeにあるものならどんな動画でも変換できます。GenYoutubeを使います。
  1. YouTubeで目的のビデオを探す。
  2. ブラウザのURLを編集する。URLにカーソルを持っていき、www.をgenに書き換える。
  3. GenYoutubeのページが表示される。下の方にフォーマットを指定してダウンロードするボタンがあるので、MP3 128kbpsかM4A 128abrのボタンを押す。
GenYoutubeサイト側でフォーマットを変換するのにしばらくかかるようですが、少し待つとダウンロードできます。

URLの編集をするのが面倒なら、ブラウザの拡張機能を追加するといいでしょう。GenYoutubeの先頭ページに方法が書いてあります。拡張機能を追加すると、YouTubeの動画の画面にボタンが追加されます。このボタンを押すとGenYoutubeのページに移動してダウンロードが可能になります。

GenYoutubeサイトにはなぜかアダルト系の広告が多い気がするので、会社で使うのは微妙かもしれません。

nice!(0)  コメント(0) 
共通テーマ:日記・雑感