23
Aug 16

Google Summer of Code 2016 – Final Report

 

Gooey with party hat, Terasology’s mascot celebrating!

Organization: MovingBlocks
Project: DAG-based Rendering Pipelines
Product: Terasology

Terasology is an open-source sandbox videogame, originally created as a Minecraft-inspired tech demo, it has grown into a powerful engine for voxel-based gameplay. The project is currently focused on architecture and extensibility, but as of early 2016 has released a first Alpha version to be a baseline engine for creating content and building gameplay.

Previously Terasology’s rendering engine was monolithic, rendering tasks were closely coupled (see diagram) and the rendering pipeline could only be altered by directly modifying the source code. Throughout summer (for my original project proposal please check here) I carefully extracted all rendering tasks, wrapped them inside into well-structured nodes and defined the OpenGL state changes that the nodes now use (diagram of the abstraction). Without changes to the visual output the engine generates, at the end of the project not only the engine can automatically reduce the number of redundant OpenGL state changes, but it can now enable third-party modules to insert their own nodes into the pipeline, something I demonstrated in the last days of the project with proof-of-concept code introducing a number of simple post-processing effects at runtime.

Finally, on final days of my project I enabled module developers to insert their own effects into the rendering engine without making substantial modifications. A video regarding to this:

Contribution Summary

Screen Shot 2016-08-23 at 05.03.15

A complete list of my commits can be found here. The pull requests that are relevant to the project:

PR ID Summary
2325  Wrapping ShadowMap and WorldReflection into a "Node"
2362  Backdrop and Skybands Nodes for DAG
2388  Post-Processing Nodes for DAG
2399  Architecture for Redundant State Elimination
2421  Finalize SetWireframe StateChange
2425  A Minor Refactor in StateChanges
2427  New StateChanges for DAG
2431  Nodes use FBOConfig to add their desired FBOs to FBM
2449  Introduces API for module developers to add graphical tweaks to their mods

The other pull requests during or before the project:

PR ID Summary
2195  Add animated water and grass options remove environment effects option
2211  Add fog volumetric fog and particle effect limit options
2286  Add auto complete for "showScreen" command
2288  Visually distinct UI elements for enabled/disabled
2316 Open up the DebugOverlay to add/remove MetricsModes

Future Plans:

  • Making module developers life easier by proposing annotations
  • There are still some nodes needs special care, planning to split them.
  • Introducing new missing state changes
  • Adding OSVR support

Acknowledgments:

It was a very enriching journey. I am very grateful to have such extraordinary mentors,  Emanuel D’Arrigo, Rasmus Praestholm and Martin Steiger. There were always there for me, like a family. 😀 I also want to thank Google for arranging this programme, without their support I wouldn’t be able to meet & work with this amazing community.


19
May 15

Using Django with Spatialite on Debian

Screen Shot 2015-05-19 at 15.21.41

SpatiaLite is a library for extending SQLite core for efficient Spatial functions. I am currently using it, during research. It’s a bit cumbersome to install it with Django 1.8. After hours of struggling, I finally installed it on my Debian system.

In order to not waste some much hours, I wrote whole process to this gist.(64 lines)

But I want to write some of gotchas here:

  • Before starting make sure you have installed build-essential.
  • Make sure you also installed python2.7-dev package, this is required for packages that uses Python.h
  • + don’t forget to install libexpat1, libexpat1-dev, both required for compiling spatialise-tools.
  • There is a slight difference with paths during the compilation. Some of the libraries tend to look only to usr/lib, usr/includebut others just only check usr/local/bin, /usr/local/include.
  • On my local system (Mac OS X), after following steps from Django’s Docs on Spatialite, I added this to my settings.py

     

17
Feb 15

Magic Tricks in iOS Development

Yes, magic :-),

I started working in a firm that develops solutions for events like congress etc. As you know, in professional life, you need to deliver your products on the time. If you are doing repetitive tasks for the products, you definitely need to do some magic tricks.

r2-d2_fixing_royal_starship

My first recommendation before telling you my magic tricks, for the iOS developers(this is true for other developers as well) is to understand requirements, priorities very well. What is the most important thing to do first? A feature or half-finished mobile app that is delivered on time.

Secondly write clean code, you know very well why. Ask Uncle Bob.

And now magic trick number one: Most of time I got frustrated by lack of automation for taking screenshots of your application inside Xcode. Well I found a workaround, which is above my expectations.

It’s called Snapshot.(I highly recommended it. Kudos to its author.) It generates the screenshots of your app by executing a UI interaction script. Author of Snapshot, Felix Krause, also has small tiny components that can be combined with fastlane to automate the whole deployment process.

I am using Swift for a while and I want to recommend you the third party libraries that I use, and all of them are magical.

  • JSON.swift, very simplistic NSJSONSerialization wrapper by Dan Kogai
  • Async.legacy, Syntactic sugar in Swift for asynchronous dispatches
  • SQLite.swift, by Stephen Celis, saves you tons of headaches if you are using SQLite in your project. Plus has a very well-written documentation.

My final advice is, even though the 3rd party libraries are tempting at first, later you can get headaches too. Please be careful while choosing a 3rd party library. It may be buggy, make sure that you really need it.

The libraries that I shared with you above, are the ones that I really like. If you find a bug in any library that you use, please don’t forget to report it or fix it and send your patch to the author.  It will be very useful for other suffering developers.

And may the force be with you!


19
Dec 14

Protocols to Closures in Swift

After releasing Socivy in a very short time, it required some refactoring in the API level. Since MVC pattern is in the foundation of the iOS development, without touching UI components, I can directly refactor only API part of the project.

Well, I highly used protocols in my implementation. For instance, if user makes a request to get available routes in the system, it uses already constructed SocivyAvailableRouteAPI to make fetch routes from the API. It handles the requests asynchronously and returns the response by calling delegate function like routeRequestDidFinish(), this is enormously helpful for me to separate controllers and models however, after increases in our API methods, unfortunately it increased protocols and unnecessary repetitions.

Not only this, increased a spaghetti code, it also affected the compile time. A big advantage was that everything was explicit, and implementation was pretty fast, it helped  our tight development schedule.

I will be adding a chatting feature for Socivy for the end of the semester for CS392 course. Before starting the feature, I thought it would be better to refactor the code and use closures and Factory and a kind of Facade pattern to reduce redundancy and decrease number of classes.

Because of the architecture of SocivyAPI security, it requires re-logins sometimes. Whenever an API request fails due to expired session, it should automatically restore and continue the calling the action and trigger the callback to update the user interface. I subclassed NSURLConnection, added completionHandler and errorHandler:

  • completionHandler: (response:NSMutableData->())?
  • errorHandler: (error:NSError->())?

Basically if request returns successfully or with an error, it will be handled by these handlers. Another problem was the data validation, NSMutableData must be a convertible to a valid JSON object. For that, I implemented a Model layer that gets exactly same handlers but this time completionHandler gets JSON as parameter. In that layer, I also handled session expires, it automatically re-logins after the validation, so no need for extra care for all methods that require authentication.

However, delegates must be removed after these, since the protocols are removed. For instance:

It is currently on ViewController side:

With closures, I think it is much cleaner than delegates, you can handle different cases, much better than delegates. On the other hand, you should never do this:

This is not very expressive, and would eventually lead to a callback hell.


24
Oct 14

On Security, Socivy

A big coincidence that last year, I wrote a post about securing a RESTful API, well this year, I experienced the same challenge, but this time in Socivy, we implemented the actual things that I mentioned in that post.

Screen Shot 2014-10-24 at 01.30.18

This semester, because of the price increases in our shuttle service, old OzU-EMS, as Socivy team, we decided to do something about it. I attended to the Student Union’s meetings, which they came up with a solution. It was making a call centre that find drivers and passengers, and match them.

We thought that it’s a very inefficient and frustrating work. On that time, it was necessary to this manually, because there were not any available solutions. At 00:30 AM, we decided why not, let’s write something. A simple car-sharing app that has the popular stops in it and can send notifications to the passenger and driver. I started the initial designs of the iOS app for it. Well, the whole development process is a long story, which we did everything that can be possibly wrong and learn it by doing. I am planning to write it whole development process in detail some time.

After finishing our web site, and hitting almost 6.000 page views, it was time to write an API for our system. Since privacy related data(phone numbers, passwords, plate number etc.) transferred throughout our system, we thought it’s necessary to take some actions for it. We put a SSL layer, both to our web page and API. However we were uncertain about deciding:

  • Saving email and password in the app or not.
  • Sending the data over HTTP parameters or in headers
  • Obfuscate or not (we are scared about decompilation of APK packages of Android)
  • Storing our API keys inside the code or in a file or in an encrypted file
  • Should we care about replay attacks which can exploit our API

Well, first thing about security, your protocols should be hidden. It’s very obvious to see that nobody knows how to do it best. By saying best, I know that it’s impossible, but we don’t want to left our front door open to bad people. For this reason, sorry I wish I could tell you all the technical details. 🙂

Even though, it is not a commercial application, but we wanted to make it in level that it can compete with them in terms of security.

  • SSL is necessary, for lot’s of reasons.
  • You should send your parameters in POST data or HTTP headers, not in parameters.
  • Obfuscate it, if you can. if you don’t want your precious algorithm to be stolen, store it to your remote server.
  • Well, it depends. You should be aware of that if someone installs your app and jailbreaks his/her iPhone. They can get access to the file that you provide. It’s better to an encrypted file, instead of putting inside the code.
  • Even if you have SSL, you can get frustrated about replay attacks. You need to find a way to eliminate that. You can put the timestamp while using the API, but it can be trivially changed by hand. I recommend you to find a way to make requests expire, you may use tokens.

Bonus: What is Socivy? How to pronounce it? (it is not sokivay, soçayvi, sochaywi)

Until next time!

 


25
Jun 14

Automating Terminal Experience with Aliases

Even though, I think terminals are kind of ancient tool, it is unquestionably useful when you are accessing to a remote server and etc. Despite the fact that I know how to add aliases for my ssh connections(thanks to Murat Kırtay), I really hated to go through to my .zshrc configuration to edit it to add an alias.

First I wrote a very basic alias adder script and placed it to the /Users/tdgunes/Utils folder. (I am sorry that I didn’t find the time to make it completely modular and clean for other people.)

After putting this into Utils folder named as new_alias.py, go to your .zshrc file. And add this at the end of the file:

Now it will become so easy to add other python files. Go to my terminal utils gist, you can download the other scripts. For instance, you can automate your ssh logins by just adding the script (named as add_ssh.py) to your aliases and then when your alias is called, by filling user name and the host name, your password twice and it will put to your keychain, so you don’t need to write your password every time when you login to a remote server.

Happy hacking!


23
Jun 14

At 2:00 AM to 4:00 AM = A Pong on Arduino

In 2013 Fall, I bought a TFT LCD Screen for my Arduino Uno (r3), in order to increase the level of fun I get from electronics. Especially after EE203 course, it is fun to play with something that works without making you crazy at the end of the day (check this hotel-safe circuit, that I built for course project. Painful hours.).

photo

I got rid of all of the cables of my hotel safe project that were attached to the breadboard. By looking this schema, I connected my TFT LCD Screen to my Arduino Uno. I tested it with TFTDisplayText example. At 2:00 AM, with an unknown inspiration, I put a small challenge to myself to make the game Pong playable throughout the serial port.

ardupong

For the code you can get it from this gist. Have fun with electronics at last. 🙂


03
Jun 14

New Lang from Apple, Swift – Making a HTTP Request

Bye bye Objective-C, welcome Swift!

Today, Apple released its new language which claims to lose C from ‘Objective-C’ and much more. I downloaded Xcode 6 Beta and tried to implement a simple HTTP Request, after reading some of their very good-written documentation about Swift. For brief features here, for iBook version here.

My first impression is, it seems like a good language, however popularity of this language will be increase not only by that, but also the great market. The reason why Objective-C is on Top 3 in TIOBE Index, my personal opinion is because of that.

Update: Well, After implementing Socivy and migrating Dilixiri to Swift, “good” seems to be not enough, for a very young language, it’s perfect. (even though, there are some small problems with UISearchBar with Scope Bar, that’s ok 🙂 )

(Apple may be inspired by Go-Lang for the type annotations.)
(Check Swift REPL mode, you will like Playground feature.)

Swift syntax seems much clear while comparing with the Objective-C version of the code above, despite its cleanness, I believe some of the core Objective-C libraries must be rewritten.

As you can see above, for making an HTTP request I have used old NSURLConnection class, and called a very sendAsynchronousRequest method. For making libraries both accessible from Objective-C and Swift, seems like Apple intention currently, so I am not expecting a huge API changes on core libraries such as UIKit etc. Apart from that, there will be dozens of new third-party libraries which will be more swiftic(like pythonic) than Objective-C ones. (For instance, HTTP library for python, requests is more pythonic than standard python library for making http requests)

For projects that are awaiting me on iOS platform this summer, it is exciting yet, a bit sad to put Objective-C behind. And yes, I think I am going to go with Swift for these projects. 🙂

Small note: My async handler unables to access to the UI, for some reason I don’t know(weirdly sometimes does). I tried to implement it by looking some of my old Objective-C implementations. Since there is nothing on the web about Swift now. (It is released just today, come on!) If you know why, it would be great if you write a comment.

Update: This is because of GCD. Please check this detailed explanation.


21
Apr 14

MongoDB Presentation for CS202

You can download my MongoDB presentation that is prepared for my CS202 course, here.

DB Presentation.001


14
Feb 14

MongoDB notes

Nowadays, I am working on an academic research, which a database server is required. Because of the amount of data we will have, and being able to easily store JSON files directly to the database, we have selected MongoDB which is a NoSQL open source database, very popular one actually,

Knowing the traditionally way to query in a database like MySQL, SQLite etc., it was a kind of a harsh learning curve for me to understand the basic functionallity at the beginning. Once I grasped that, learning other things are significantly faster than the beginning.

So, there are some small notes that I gathered from lot’s of different sources that I wanted to share.

For me handling and designing data is like a medical surgery. Let’s assume you have an increasing amount of data, records increases over hundreds of thousands. While recording the data, data source may give some fields corrupted or not the way that you expect, even you have read the all things on documentation of that source. (There is a very small percentage of finding a well-documented data sources, if you think you found, it may also not always perfect.) For myself, I was fetching a data which has geolocation features(latitude, longitude) for more than one source. I used inheritance and abstraction for creating a base point, and implemented new classes which inherits this abstract class to convert these data based on differences.

However later implementing these services, after several days I wanted to make 2dsphere based queries on Mongo to search records that are close to my-desired-location. It didn’t worked. So I figured out that, that source was the only one that gave coordinates in String form. Hopefully, I didn’t have so many records. I simply write this to the MongoDB console and magic happened! And I changed my subclass immediately. I learned that I should not get lost in the joy of sending a JSON file without looking the data. Testing, testing, evaluating results and testing again is very important.

After for making queries like, “get me records that my distance to these records are at max is this ..” were easy. (Note that $maxDistance is always meters for $geometry = GeoJSON format else radians)

Before writing this query, you must make sure that you have indexed this “loc” field as “2dsphere”. Indexing is required for this action.

Other handful notes:

This one says, find me 5 items where “city” field is Istanbul and print prettily.

This case, you may have a customer data, you expected customers to enter two mails or write only a primary mail. Code snippet at below, gets all customers’ primary and secondary mail fields who actually have “primary mail” field or “secondary mail” field. Making logical operations like this are very powerful and save your time.

You wanted to get rid of a field, that is easy:

For more details about writing a query you should also check this page.

I wished that you could have written more about MongoDB and of course using it with a Python Adapter. May be next time.