tag:blogger.com,1999:blog-70528135810990809532024-03-05T21:24:33.226+01:00Rom's Code CornerAnother blog about software development, python, agile, crafsmanship.Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comBlogger183125tag:blogger.com,1999:blog-7052813581099080953.post-83460697937848403912019-01-07T21:49:00.002+01:002019-01-07T21:49:52.248+01:00VueJs for the skeptic<p>Initially, I wanted to write a long article about my skepticism when hearing about shiny piece of tech, especially in the frontend world. I’m very tempted to reject all these frameworks and tools every cool kid is using these days.</p>
<p>Fortunately, I did not and I started using VueJs with Vuex last year. The advantage of all these frameworks (Vue, React, Angular,…) is that they do the DOM and events manipulation for you. To do so, they use an implementation of the good old observer pattern: every change of data in a component or in the global state can be captured so that your UI is automatically updated. They (at least React and Vue) use a DOM abstraction (the <a href="https://stackoverflow.com/a/21965987/1536219" title="Virtual DOM">virtual DOM</a>) to do this more efficiently than you, I’m curious to see how it works in the source code!</p>
<p>On the other hand, be aware that observed state is not made of plain Javascript objects, but object proxies. Not all action of on the proxy will <a href="https://vuejs.org/v2/guide/list.html#Mutation-Methods" title="Proxy action">trigger an update of the observers</a>. Moreover, some of the dependencies you may use can be affected by the way the DOM is manipulated. For instance, with Vue you may want to use <a href="https://github.com/FortAwesome/vue-fontawesome" title="vue-fontawesome">Font Awesome Vue component</a> instead of the standard Font Awesome JS bundle. I spent hours to understand why my icons were not updated in some cases!</p>
<p>So, when do you want to use a front end framework? When you plan to update your UI in response to data update and you don’t want your UI logic to be mixed with element creation and event handling. And that’s pretty often!</p>
</body>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-40338922925644744752018-06-16T16:00:00.000+02:002018-06-16T16:00:15.094+02:00Programming with dates is hard
<p>Time and date is not a trivial problem in programming. You can expect different precision, handle different timezones and different format when you convert them from and to a string.</p>
<p>You may remember some issues related to date that appended in the IT world. Take <a href="https://en.wikipedia.org/wiki/Year_2000_problem" title="Y2K Issue">Y2K</a>, that obliged companies to check and fix the way they handled date in their program before the millennium. I guess it's been a costly procedure in some of the case in term of research, fix and testing. For non tech people though, it's a ridiculous case: in our daily life, we use time and dates casually without thinking about it.</p>
<p>Another sign that handling dates is not trivial: the plethora of library available. In Python, the standard module <em>datetime</em> does not handle time zones, so we have <a href="https://pypi.org/project/pytz/" title="pytz">pytz</a>. But as it's not human friendly, everyone got to use <a href="https://pypi.org/project/arrow/" title="Arrow">Arrow</a>. Then <a href="https://pypi.org/project/pendulum/" title="Pendulum">Pendulum</a> came out, claiming that <em>arrow</em> got it wrong and it became the new standard for community. Meanwhile, Kenneth Reitz (requests, pipenv,…) released <a href="https://pypi.org/project/maya/" title="Maya">Maya</a> to do the same thing. Help, we're lost!</p>
<p>It's not specific to Python. In Java, <a href="http://www.joda.org/joda-time/" title="Joda-time">Joda-Time</a> has been the preferred library to handle dates, rather than the standard library. Then Java 8 came out with a new java.time package that can be used instead of Joda. I worked recently in a code base that mixed both libraries. It inspired me this post.</p>
<p>On the other hand, when you have chosen a library, getting the current date is super simple, using a call like <code>mytimelib.today()</code>. Cool. How about testing then?</p>
<p>Actual time does not belong in your system, it's a dependency, bound to the real world. If your business domain relies on date for some sort, you may avoid to use your date library directly.</p>
<p>When I need to integrate business logic based on date, I use my own Date and Calendar classes which allows me to:</p>
<ul>
<li>Get the precision I want for comparison. For instance, my domain can rely on lapsing dates with month precision and I don't need more.</li>
<li>Have a reference implementation. My dates can be created from instances created with different date libs.</li>
<li>Get default formats for date as String</li>
<li>Create a fake calendar for testing that gives me control on current date generation. So I do not need to scratch my head when I have to compare dates in tests.</li>
</ul>
<p>In the past I worked on a project for an insurance company. We naively used <code>mylib.today()</code> calls all over the place. When the main features were implemented and tested, business people told us that they wanted to test the system behaviour when contracts get on term. To do so they wanted to be able to alter system current date.</p>
<p>Guess who replaced all the calls by injecting a custom Calendar and provided a service endpoint that allow users to modify the current date for testing…</p>
<p>So pay attention to your business domain. If it relies on dates in some sort, you'd better treat them as an external dependency. Then you'll be able to use the lib you want, and even to change it seemlessly if it get oldfashionned.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-19522456320367325142018-06-10T17:56:00.001+02:002018-06-10T22:09:30.517+02:00My typing is bad (but getting better, thanks for asking)
<p>I've always found I'm bad at typing. I make a lot of typos, my speed is not that fast and I don't manage to use all my fingers.</p>
<p>Recently, I have been working on a Java code base and I got very frustrated by the fact I'm a slow typer!</p>
<p>As my productivity is bound to a typing speed that could be improved, I decided to train myself to touch typing. It's a bit frustrating for now because my speed is about 40 wps when I'm training on <a href="https://www.gnu.org/software/gtypist/" title="gTypist">gtypist</a>. But I'm training every day, and even if my improvements are not as fast as I'd like, I have to admit I'm already a better typist:</p>
<ul>
<li>I can now type with my ten fingers,</li>
<li>I discovered that I can use the left shift key,</li>
<li>I can type without looking at the keys with a decent accuracy.</li>
</ul>
<p>Till now, I was using tricks to compensate my lack of speed :</p>
<ul>
<li>Using <a href="http://www.vim.org" title="Vim">Vim</a>,</li>
<li>Switching to QWERTY layout though I'm French: the accessibility of keys [, ], { and } on the AZERTY keyboard is soooo painful!</li>
<li>Adopting a programming language that requires less typing (Python vs Java)</li>
</ul>
<p>Though being a proficient developer doesn't rely only on typing speed, typing faster maximizes the throughput between idea and code.</p>
<p>I'm still far from Gary Bernhardt. Seriously, what is this guy typing speed? 120 wpm? It's insane!</p>
<p>I'm also thinking about investing into a mechanical keyboard. Maybe an ergonomic one, like the <a href="https://ergodox-ez.com/" title="Ergodox EZ">Ergodox EZ</a> or the <a href="https://shop.keyboard.io/" title="Keyboardio">Keyboardio</a>.</p>
<p>Edit 1: according to <a href="https://twitter.com/fabi1cazenave" title="fabi1cazenave on Twitter">@fabi1cazenave</a>, the Keyboardio is far better than the Ergodox and French people should use the <a href="https://qwerty-lafayette.org/" title="QWERTY Lafayette layout">QWERTY Lafayette</a> layout.</p>
<p>Edit 2: ten years ago, Steve Yegge wrote this <a href="http://steve-yegge.blogspot.com/2008/09/programmings-dirtiest-little-secret.html" title="Programming's dirty little secret">nice article</a> in favor of touch typing</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-63385700220526197122018-05-24T09:03:00.000+02:002018-05-24T09:03:17.226+02:00Advanced scraping
<p>I wrote web scraping scripts lately for a client to download invoices. I really like web scraping because it’s an automation that allows to save a lot of time and it often demands to reverse engineer a site. Just like a real hacker, ma! Here are some techniques I use.</p>
<p><strong>Disclaimer:</strong> using a robot to scrape a website can be prohibited by its owner, so check that you stay in the terms of usage conditions and please have a fair behaviour.</p>
<h1 id="tooling">Tooling</h1>
<p>Mostly, I use Python 3, Requests and BeautifulSoup. Everything can be installed with Pip.</p>
<h1 id="session">Session</h1>
<p>Send all your requests through a <a href="http://docs.python-requests.org/en/master/user/advanced/#session-objects" title="Session in requests">Session</a>. It has several advantages:</p>
<ul>
<li>it handles session cookies for you,</li>
<li>it keeps the TCP sockets open,</li>
<li>it allows you to set default header for all the requests you’ll do. For instance, to setup the user agent header in a session.</li>
</ul>
<p>Like this:</p>
<pre><code># Wow, the UA is quite old!
session = requests.session()
session.headers.update(
{
"User-Agent": (
"Mozilla/5.0 (X11; Linux x86_64; rv:58.0)"
"Gecko/20100101 Firefox/58.0"
)
}
)</code></pre>
<h1 id="authentication">Authentication</h1>
<p>When you want to gather data from the Internet, web sites will often require you need to sign in. Mostly, you’ll have to send your credentials within your session in a HTTP POST request:</p>
<pre><code>session.post(your_url, data=form_data)</code></pre>
<h2 id="csrf-protection">CSRF protection</h2>
<p>A good practice when handling forms in a website is to associate them with a random token. Hence you have to load a the form from the website before submitting it. That’s basicly <a href="https://www.owasp.org/index.php/Cross-Site_Request_Forgery_(CSRF)" title="CSRF on OWASP wiki">CSRF</a> protection, to avoid a malicious site to make use of an existing session cookie to discretly perform post requests.</p>
<p>It’s a general good practice, though it’s questionnable on authentication form!</p>
<p>By the way, when authenticating, you’ll often need to load the sign-in page, gather all the input fields then update them with username and password. Something like this:</p>
<pre><code>soup = BeautifulSoup(self._session.get(self._url).text, "html5lib")
login_data = {}
for input in soup.find_all("input"):
if input["type"] in {"hidden", "text", "password"}:
login_data[input["name"]] = input["value"]
login_data["credential"] = my_username
login_data["password"] = my_password</code></pre>
<h2 id="saml">SAML</h2>
<p>Sometimes, the authentication process is complicated. For a website, I needed to pass through a <a href="https://en.wikipedia.org/wiki/Security_Assertion_Markup_Language" title="SAML authentication protocol">SAML</a> authentication protocol.</p>
<p>SAML authentication consists in several exchanges between client and server to generate the authentication token. The token has to be extracted from the redirection history.</p>
<p>To do so, requests lib allows you to browse the redirection history from a response and update the session cookies.</p>
<p>In requests, access the response history with:</p>
<pre><code>response.history # get the response chain has a list</code></pre>
<p>You can then get the cookies from one of the responses as a dict:</p>
<pre><code>saml_response_cookies = requests.utils.dict_from_cookiejar(
response.history[1].cookies
)</code></pre>
<p>Finally, update your session cookie jar with these cookies:</p>
<pre><code>session.cookies.update(saml_cookies)</code></pre>
<h1 id="the-information-is-not-in-the-source-code">The information is not in the source code</h1>
<p>Classic web scraping works well when all the content of the page is sent in the html on page load. What if it is a Single Page App on which all the content is loaded dynamicly in JavaScript?</p>
<p>Look for XHR (use of AJAX) in the "Network" tab in the browser’s dev tools. You can then replay these XHR directly with <em>requests</em> and parse the response.</p>
<p>Most of the time, the response will be a JSON document. It’s good news, as parsing JSON content is far easier than parsing HTML. Just use <em>json</em> module from Python stdlib.</p>
<p>Sometimes the requests are built by the JavaScript code. I once scraped a website that converted session information in base64 to build a request QueryString in Javascript. By chance, these functions where readable, so I reimplemented them in my Python code.</p>
<h1 id="i-dont-see-the-whole-document">I don’t see the whole document!</h1>
<p>Sometimes, beautifulsoup silently fails to parse all your HTML document. It’s really annoying because you often spend a ridiculous amount of time to figure out that the error comes from beautiful soup parsing.</p>
<p>Try to vary parsers with beautifulsoup. Using <a href="https://pypi.org/project/html5lib/" title="Html5lib on PyPI">html5lib</a> gave me the best results. I think you can add this module directly with you dependencies and use it by default:</p>
<pre><code>soup = BeautifulSoup(content, "html5lib")</code></pre>
<h1 id="when-everything-fails">When everything fails</h1>
<p>Sometimes though, reverse engineering how a website works is too time consuming (JavaScript is minified, lot of state information is kept on client side,…). I then switch to headless browser automation. I do so with Chrome with headless option and <a href="https://pypi.org/project/selenium/" title="Semenium package on PyPI">Selenium</a>.</p>
<p>This is a heavier approach as you need to script all the actions which would be performed by a real user: loading a page, waiting for all the content to be loaded, fill in form fields, click on buttons.</p>
<p>Usually, I fire up Chrome without the headless option during the script writing.I switch back to headless when everyting works.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-67449923346432989412018-04-07T22:10:00.001+02:002018-04-07T22:10:28.962+02:00Download files with Python, Selenium and Chrome headless<p>I have recurring tasks these days that consist to automatically download files on the Internet. Usually, <a href="http://docs.python-requests.org/en/master/" title="Awesome request lib">requests</a>, <a href="https://www.crummy.com/software/BeautifulSoup/bs4/doc/" title="BeautifulSoup">beautifulsoup</a> and some tricks do the job effectively. Sometimes though, I have to play it hard and ask <a href="https://www.seleniumhq.org/projects/webdriver/" title="Selenium">Selenium</a> and Chromium* headless to do the heavy lifting. Alas, asking Chromium to automatically download files is not clear.</p>
<p>I found the solution in <a href="https://bugs.chromium.org/p/chromium/issues/detail?id=696481" title="Chromium tracker">Chrome tracker</a>, and you read it bellow written in Python:</p>
<pre><code>from selenium import webdriver
# Let's create some option to make Chromium go headless
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('disable-gpu')
# Launch the browser
browser = webdriver.Chrome(chrome_options=options)
download_dir = tempfile.TemporaryDirectory().name
os.mkdir(download_dir)
# Send a command to tell chrome to download files in download_dir without
# asking.
browser.command_executor._commands["send_command"] = (
"POST",
'/session/$sessionId/chromium/send_command'
)
params = {
'cmd': 'Page.setDownloadBehavior',
'params': {
'behavior': 'allow',
'downloadPath': download_dir
}
}
browser.execute("send_command", params)</code></pre>
<p>There you go, happy scraping!</p>
<p>* <em>Of course, it works with regular Chrome too!</em></p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-74328032830132013812018-01-14T07:57:00.000+01:002018-01-14T07:57:20.622+01:00Some talks<p>I'm watching tech talks less often these days. Most of the time, the talks are too long (45+ minutes). Nowadays, I don't even try to watch a talk about new piece of software or framework: if I want to spend my energy on these topics, I'd better play with an online tutorial.</p>
<p>Here are two talks videos I watched and liked lately. First one is <a href="https://youtu.be/f84n5oFoZBc" title="Hamnock Driven Development by Rich Hickey">Hammock Driven Development</a> by <strong>Rich Hickey</strong>. He is the creator of <a href="https://clojure.org/" title="Clojure language website">Clojure</a> programming language. In this talk, he explores the fact that we have 2 modes of thinking, the first in <em>conscious</em> and the second is <em>unconscious</em> and work by itself. It's about these "Eureka" moment that we experience as developers when we are away from the keyboard. You'll see why the hammock is a smart choice to enforce unconscious thinking!</p>
<p>The <a href="https://www.youtube.com/watch?v=GRr4xeMn1uU" title="Greg Young - Stop Over-Engineering">second talk</a> is by <strong>Greg Young</strong>, who is known for his work on CQRS and Event Sourcing architecture patterns. He demands us to stop over engineering. It's OK not to automate all the stuff by programming. It's about the struggle the desire to show the power of our programmer brains by building over complicated frameworks.</p>
<p>Good watching!</p>
<p>PS: if you think videos on Youtube still last too long, you can increase or decrease speed with shortcuts <em>shif+.</em> and <em>shift+,</em>, look at <a href="https://sites.google.com/a/umich.edu/going-google/accessibility/google-keyboard-shortcuts---youtube" title="Youtube shortcut">this page</a> to a more complete list of shortcuts.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-25347892832726976942018-01-07T08:15:00.000+01:002018-01-07T08:15:14.230+01:00First months as a freelancer<p>I realize I hardly take time to write these days. Actually, I've had a lot to do for the last few months, with my activity as a freelancer to build, the work in my house and my family to take care of. But I'm having a great time!</p>
<p>I quit my salary job last summer to start my own business as a freelancer. I barely had a professional network, just one or two contacts that lead to nothing. So I created a profile on various freelancer platforms. And it worked! Even if what I earn is bellow my goals, I think it's a good start.</p>
<p>Currently, I'm focusing on short missions. I did not want to go full-time at the beginning since I wanted to be present for my family after more than 10 years working a lot. I also wanted to start slowly, because I ended my previous job quite unmotivated, and basically I wondered if I still wanted to do programming for a living.</p>
<p>And I found some missions, I realized the job on time and clients have been quite happy with what I delivered. That's a good point. Here is what I have done so far:</p>
<ul>
<li>A script to save NASDAQ stock events in a database and to perform simple analytics on it,</li>
<li>Writing of functional specifications an mock-ups in <a href="http://url" title="Baslsamiq Mockups">balsamiq</a>,</li>
<li>Add features for a wysiwyg editor for a printing company,</li>
<li>Some web scrapers to automatically retrieve invoices from websites.</li>
</ul>
<p>I'm glad because I worked with Python and JavaScript, which is what I want to become specialized in. It's not much but I learned a lot:</p>
<ul>
<li>I can find missions and get paid.</li>
<li>I do great job.</li>
<li>I like it.</li>
<li>It's not a big deal if I don't get all the missions I apply for. There will be other opportunities.</li>
</ul>
<p>So to me it's a good start and I hope to work more in 2018, to meet great people, and get decent revenue from this!</p>
<p>So, do I still want to write software for a living? Of course I do.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-88449668492040370262017-11-20T00:28:00.000+01:002017-11-20T00:28:09.291+01:00Poor man's pomodoro<p>It seems that I have not written about it until know: I have to confess I'm a big fan of the <a href="https://cirillocompany.de/pages/pomodoro-technique" title="Official website, in case you want to buy something">Pomodoro Technique</a>.</p>
<p>Now that I have a new laptop, I've been looking for a timer software. On Windows and MacOs, I used <a href="http://tomighty.org/" title="Tomighty">Tomighty</a>. Cool, but Qt version is not in Debian repo and Java version… well it runs on the JVM!</p>
<p>Then I learned about a magical command, <a href="https://ss64.com/bash/notify-send.html" title="Notify-send man page">notify-send</a>, which sends desktop notifications. So here we go, my timer will be super simple and written in shell script:</p>
<pre><code># pomodoro aliases
alias PS="pomodoro_start"
alias PSB="pomodoro_short_break"
alias PLB="pomodoro_long_break"
# launch a 25 minute pomodoro
pomodoro_start() {
# I write the start time, in case I miss the notification
echo "pomodoro start `date +%H:%M:%S`..."
sleep 1500
# using critical level of urgency force me to click on the notification
# pop up to dismiss it.
notify-send --urgency=critical "Poor man's pomodoro" "Time for a break"
# cvlc is VLC CLI. I play a sound to help me to get out of the zone.
cvlc --play-and-exit --quiet /some/sound.mp3 > /dev/null 2>&1
echo "pomodoro end `date +%H:%M:%S`..."
}
pomodoro_short_break() {
echo "pomodoro break `date +%H:%M:%S`..."
sleep 300
notify-send --urgency=critical "Poor man's pomodoro" "Break over, go back to work"
cvlc --play-and-exit --quiet /some/sound.mp3 > /dev/null 2>&1
echo "pomodoro end `date +%H:%M:%S`..."
}
pomodoro_long_break() {
echo "pomodoro long break `date +%H:%M:%S`..."
sleep 900
notify-send --urgency=critical "Poor man's pomodoro" "Long break over, go back to work"
cvlc --play-and-exit --quiet /some/sound.mp3 > /dev/null 2>&1
echo "pomodoro end `date +%H:%M:%S`..."
}</code></pre>
<p>The Pomodoro Technique is far to make <a href="https://www.infoq.com/news/2010/02/pomodoro-critique" title="Critique">unanimity</a>. I do not practice it <em>strickly</em> by the book. What I like is that it forces me to get out of the zone when I work on a long running task. Though being the zone and experiencing the flow feels good, relying on my timer helps me to get out of it, take a step back from your work and think. Does what I'm doing right know really worth it?</p>
<p>Contrary to what is usually told about the flow, I don't think it's more <em>productive</em>. It's just pleasant. So while it's a good thing for your hobbies to gain the most of energy out of them, it may not be so appropriate when you have stuff to do at work.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-13606557022698577812017-10-11T09:20:00.001+02:002017-10-11T09:20:20.324+02:00Why I chose GNU Linux for my professional laptop
<p>Before you ask the question, or even care about it, here is why I chose to put <a href="http://romscodecorner.blogspot.fr/2017/10/debian-on-my-new-thinkpad-t470p.html" title="Previously in code corner">Debian</a> on my new laptop.</p>
<p>I have to say I really do not like Windows. I'm sure the system got better and better among its versions and that Microsoft itself is improving its opensource policy. However, I do not like how Windows looks, I do not like how it works, I feel in jail with it. In my previous job, my screen used to show a terminal with Cygwin and Vim most of the time, helping me to gain a nice nerd reputation!</p>
<p>The classic dilemma then is to choose between buying a Macbook, or install Linux an another laptop.</p>
<p>We have a 2013 iMac at home and I was globally happy with it at first. Everything is simpler with it, the system is stable, the hardware is beautiful. But with the updates, some stuff got annoying:</p>
<ul>
<li>The OS randomly refuses to mount my USB keys</li>
<li>Launchpad randomly refuses to allow me to search apps with the keyboard</li>
<li>The system takes longer and longer to boot</li>
<li>The system won't launch graphical app (even the Finder) from within tmux</li>
<li>…</li>
</ul>
<p>Moreover, last releases of Macbooks are sooo expensive, and lots of people complain about hardware quality on Twitter.</p>
<p>I started regretting my previous experience with Debian on my previous PC. On the other hand, I know that with Linux, I can spend a lot of time trying stuff and fix (sometimes non existent) issues.</p>
<p>So I chose to buy a Lenovo ThinkPad, because I saw at Pycon that a LOT of people had one. Chances are that they are Linux friendly (besides, I think I've read somewhere that they're used by people from Red Hat).</p>
<p>I chose Debian because of the <em>free software</em> philosophy behind it, the large choice of packages and the fact that private software are often distributed as Debian packages. I hesitated with Ubuntu. But seeing Amazon ads on the default desktop on Ubuntu decided me.</p>
<p>I completed the installation two weeks ago and everything goes surprisingly well, far beyond my expectations.</p>
<p>For now, I keep the dual boot with Windows, in case I have to use it for work. But I hope to drop the partition soon!</p>
<p>If you're questioning yourself about going on Linux for your professional laptop, here are some advices.</p>
<ul>
<li>Consider buying a Lenvo Thinkpad. They are well supported and you have a customer support. I read complains about Dell XPS, though they are shipped with Ubuntu. I also saw people with Asus Zenbooks, it can be a good alternative.</li>
<li>You'll have software and hardware issues for sure. Yet, during my Linux year I always found a solution eventually on the Internet. In the extreme case, you still can boot on the latest release of the kernel. It's not that hard.</li>
<li>You can easily taylor a system that fits your needs and helps you to be more efficient, it's priceless when you rely on your laptop to make money.</li>
</ul>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-85450655297265733962017-10-07T16:14:00.000+02:002017-10-07T16:14:28.104+02:00Debian on my new ThinkPad T470p<p>As I started my new activity as a contractor, I bought a new laptop: a Lenovo Thinkpad T470p. Not that I fell in love with Windows, but to install <a href="https://www.debian.org/" title="Debian distribution home page">Debian</a> on it. Here is the description of the <em>journey</em>. I leave the reason of my choice in another post.</p>
<h1 id="hardware">Hardware</h1>
<p>The hardware is made of:</p>
<ul>
<li>A i7 processor</li>
<li>8 GB Ram</li>
<li>256 GB Samsung SSD</li>
<li>A Nvidia GeForce 940 mx graphic card</li>
<li>Intel 8265 wifi component</li>
</ul>
<p>I'm quite happy with this.</p>
<h1 id="what-youll-need">What you'll need</h1>
<p>Prepare this stuff or it won't work:</p>
<ul>
<li>Your laptop</li>
<li>An USB key (not a big one, I used a 1 GB old key I had)</li>
<li>An Internet connection through ethernet cable, since what I describe won't work by wifi.</li>
</ul>
<h1 id="installation">Installation</h1>
<p>I have several installation of Linux behind me, but only on BIOS based systems and raspberry PI. So I learned about UEFI, not without some difficulties!</p>
<p>I want to install Debian testing, because Debian stable usually contains old stuff. So at the time I write these lines, its little name is <em>Buster</em> and it might be Debian 10th release.</p>
<p>First thing to do is to download an ISO. Do yourself a favour and don't chose a Debian testing installer. It's an alpha version and will simply not work (I used netinst ISO and it doesn't contains the mandatory packages). Go on with stable distribution. I chose netinst, because I did not have a big bootable USB stick at home and I want a minimal installation. I chose an amd64 architecture (though you have an Intel CPU… I know, I know…).</p>
<p>Now you have to make an UEFI bootable USB key. To do so, from Windows 10, I used <a href="https://rufus.akeo.ie/" title="Rufus utility">Rufus</a>. I first made the mistake of choosing to make it BIOS bootable. The thing is that depending the type you choose (UEFI or BIOS), the installer will now if it has to install on UEFI or BIOS. I made the bad choice at first and the laptop did not boot at the end of the installation. So, <strong>choose to format the key as a UEFI boot device</strong>.</p>
<p>Remember to shrink your disk to have some room for your Linux system. I used the disk management tool provided with Windows 10. I did not gain lot of disk space, but I guess it is the most secure way to do so.</p>
<p>Now reboot having the key plugged and go into the BIOS by pressing Return while booting. In there, <strong>disable Secure Boot</strong> in security options (I guess). You can also change boot order to make it boot directly from the disk, instead of Windows bootloader. Exit the BIOS and press F12 to access the list of devices to boot from. Choose the USB stick. You now enter the installer.</p>
<p>From here, you won't face a lot of trouble, do whatever you want. Just notice:</p>
<ul>
<li>I prefer the simple ncursed based installer instead of the graphical one. I know I'm a nerd but believe me, it's easier to use.</li>
<li>The wifi chip needs a non free firmware that is not provided by the ISO. The installer propose you to get it from another support. Say no.</li>
<li>I advise you to keep the installation to its minimum, without a graphical desktop since you will upgrade to testing right after the process.</li>
<li>I chose not to use Swap partition. Come on, I have 8GB RAM! It's controversial though, and I could create a swap file later on if things goes wrong.</li>
</ul>
<p>At the end of the installation, remove the USB key and let the installer reboot the machine for you. You should then see Grub to boot on your Debian.</p>
<h1 id="post-installation">Post installation</h1>
<p>First thing I did was to switch to <em>testing</em> repositories and activate contrib and non-free repo. To do so, replace the content of <code>/etc/apt/sources.list</code> with:</p>
<pre><code>deb http://ftp.fr.debian.org/debian/ testing main contrib non-free
deb-src http://ftp.fr.debian.org/debian/ testing main contrib non-free
deb http://security.debian.org/debian-security testing/updates main contrib non-free
deb-src http://security.debian.org/debian-security testing/updates main contrib non-free
deb http://ftp.fr.debian.org/debian/ testing-updates main contrib non-free
deb-src http://ftp.fr.debian.org/debian/ testing-updates main contrib non-free</code></pre>
<p>Then, as root, launch the upgrade:</p>
<pre><code>apt-get dist-upgrade</code></pre>
<p>Remember to install the wifi firmware with:</p>
<pre><code>apt-get install firmware-iwlwifi</code></pre>
<p>And <em>voilà</em>, you can install whatever you want.</p>
<h1 id="troubleshooting">Troubleshooting</h1>
<p>I faced a few caveats after the installation, partly because I chose to use <a href="https://xfce.org/" title="XFCE4">XFCE</a> as my desktop environment and <a href="https://freedesktop.org/wiki/Software/LightDM/" title="Lightdm">lightdm</a> as a display manager (because I like to try different desktop).</p>
<p>The main problem was the inability to log back to my session after locking it. After spending a looong time on Google, I found that it is due to <strong>an incompatibility between XFCE4 power management and lightdm-locker</strong>, the locking frontend for lightdm. The no brainer here is to install and configure the (ugly) xscreensaver, as XFCE tries to use it before light-locker while locking the session. Its done in a simple shell script (/usr/bin/xflock4), so feel free to check and hack it if you want.</p>
<p>I also found some error messages related to firmware in the syslog, which made me install <em>firmware-misc-nonfree</em> package. I thought at first it was related to the locking problem, but actually I do not know the effect of this fix.</p>
<h1 id="conclusion">Conclusion</h1>
<p>I'm really happy with this system. It's amazingly fast. I took time to install Node and Python 3.6 from sources and I'm ready to hack!</p>
<p>If everything works well in near future, I may drop Windows partition and reinstall the system to use the whole disk and encrypt the partitions. The journey is far from being over!</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-9492254562579707112017-09-22T00:30:00.000+02:002017-11-06T00:04:15.647+01:00Adopting vim native package management.<p>OK, today, I switch to Vim 8 native <a href="http://vimhelp.appspot.com/repeat.txt.html#packages" title="Vim package help page">package</a> management.</p>
<p>Before, I used <a href="https://github.com/tpope/vim-pathogen" title="Tim Pope's Pathogen repository">Pathogen</a> with controversial git <a href="https://git-scm.com/book/en/v2/Git-Tools-Submodules" title="Git submodule">submodule feature</a>.</p>
<p>I think it's always better to use native features instead of third party ones, even if I never had any issue with <em>Pathogen</em>.</p>
<p>If you have a similar configuration, just type following commands to switch (assuming your modules are in .vim/bundle directory)</p>
<pre><code>cd ~/.vim # go to your vim config directory
mkdir -p pack/myplugs/start # create a pack dir for plugin loading on startup
for DIR in bundle/*; do git mv $DIR pack/myplugs/start; done # move every submodule from bundle to pack keeping git aware </code></pre>
<p>You may have to update your git submodules if they emptied in the process:</p>
<pre><code>git submodule update --recursive --remote</code></pre>
<p>Then edit your ~/.vimrc file to remoce call to pathogen</p>
<pre><code>" remove this line
call pathogen#infect() </code></pre>
<p>Rerun vim, to check if everything is OK for you. Cleaning the .vim directory is left as an exercise.</p>
<p>Native package management feature adds plugins to the runtime path at startup for bundles stored in <code>.vim/pack/whatever/start</code>. You can also configure optional module by putting them in <code>.vim/pack/whatever/opt</code>. If you put a module <em>foodebug</em> in that directory, you have to activate it manually with command:</p>
<pre><code>:packadd foodebug</code></pre>
<p>You could achieve the same with <em>Pathogen</em> though.</p>
<p>Unfortunately, using packages loaded at vim startup does not make their documentation directly available with <code>:h</code> command. To do so, it seems you have to manually update your help tags using:</p>
<pre><code>:helptags ~/.vim/pack</code></pre>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-57060302749191551692017-09-20T10:46:00.002+02:002017-09-20T10:46:45.120+02:00Recaptcha in Django<p>I've been writing a small website with <a href="https://www.djangoproject.com/" title="Django project homepage">Django</a> these days. I wanted to put a form with a captcha to prevent an evil hacker from exploding my database quotas. So I looked at the popular Google's <a href="https://developers.google.com/recaptcha/old/intro" title="Recaptcha">Recaptcha</a> service.</p>
<p>According to documentation, when processing the form, you have to send a token to Google and analyze the response to actually validate the form. Well, OK, it's not so complex, but I expected a 2 minutes solution!</p>
<p>Fortunately, Django is easily extensible through third party apps. And <a href="https://pypi.python.org/pypi/django-recaptcha/1.3.1" title="django-recaptcha">an app</a> exists for my reCaptcha purpose.</p>
<p>So the message here is: if you want to add an external service to your website, look at what Django has for you or if a Django app exists for that purpose.</p>
<p>You (are old and you) want to add an RSS / Atom feed? It <a href="https://docs.djangoproject.com/en/1.11/ref/contrib/syndication/" title="RSS feed provided by Django">comes directly with Django</a>!</p>
<p>More on reCaptcha. If your want to see the current reCaptcha widget to let Google guess automagickly if you're a robot, add the option bellow to your <em>settings.py</em>:</p>
<pre><code>NOCAPTCHA = True</code></pre>
<p>And a last word about privacy. Google's recaptcha uses <a href="https://en.wikipedia.org/wiki/DoubleClick" title="DoubleClick page on wikipedia">DoubleClick</a> and the fact that Google knows everything on everyone. I chose that because I believe that users are used to it and because of its simplicity. If privacy is an issue for you (it's legitimate), other kind of captcha exists: color based, math based, etc.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-8290812210287952072017-07-09T13:32:00.000+02:002017-07-09T13:32:15.668+02:00Feedback loop please<p>I’ve been working on data transfer processes these days. The transfer time is ridiculously slow regarding the amount of data (about 100k records). By the way it is geographic data, so it might be normal. It irritated me though and made me think about feedback loop.</p>
<p>The importance of shortening the feedback loop between coding and watching the result is huge.</p>
<p>A short feedback loop helps you to achieve more in less time. It also prevents you to get bored and fall into procrastination.</p>
<p>Days spent waiting for feedback are long and make me feel depressed. Days when I get quick feedbacks are pretty cool, whatever I am doing.</p>
<p>Some idea to shorten feedback loop :</p>
<ul>
<li>TDD of course makes you write code that can be tested faster. Actually, taking much time to write test or wait for the test run to finish is a sign that you do something wrong. However TDD is not sufficient to test your whole app and you have to add system tests. They last longer.</li>
<li>Avoid network. Your network is unreliable. Use a local database on your machine when it’s possible. It also will prevent to crash the shared database for respect to your colleagues.</li>
<li>Automate what can be: to load your dataset, check the initial state of the system, test, cleanup.</li>
<li>Reduce the amount of data you will handle during your tests, at least for early stage of development</li>
<li>Ask for faster hardware in <em>last</em> resort.</li>
</ul>
<p>This blog post is can of a truism. At least it will remain a note to myself!</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-31413863599655479582017-04-14T00:05:00.002+02:002017-04-14T00:05:57.447+02:00logging.exception
<p>By skimming Python documentation today, I realized that <em>logging</em> module provides an <a href="https://docs.python.org/3/library/logging.html?highlight=logging.exception#logging.exception" title="logging.exception in Python documentation">exception level</a>. Its goal is to show you a stacktrace when you call it from an except block.</p>
<p>You may have noticed that if you log an exception with level <em>error</em>, you only get exception message:</p>
<pre><code>#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import logging
def send_unexpected_exception():
raise Exception("I did not expect this")
logging.basicConfig(level=logging.DEBUG)
try:
send_unexpected_exception()
except Exception as e:
logging.error(e)</code></pre>
<p>Returns:</p>
<pre><code>ERROR:root:I did not expect this</code></pre>
<p>Now use exception level:</p>
<p>It returns:</p>
<pre><code>ERROR:root:I did not expect this
Traceback (most recent call last):
File "logging_exception.py", line 12, in <module>
send_unexpected_exception()
File "logging_exception.py", line 7, in send_unexpected_exception
raise Exception("I did not expect this")
Exception: I did not expect this</code></pre>
<p>I think it’s an interesting way to provide feedback and context if your app raises an unexpected exception.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-5673711732737950212017-01-18T23:14:00.001+01:002017-01-18T23:14:22.792+01:00Accept my ideas<p>I like to question everything when I land into an existing team. As a normally pretentious person, I like looking smart, making people feel I'm experienced and that they can count on me. Yet, when I say what I find bad in a code base and when I expose my brilliant ideas, my new teammates don't follow me. That hurts my pride so hard!</p>
<p>“we've done this until now, there is no reason to change” says the lead dev. Of course they are reasons change, since the need for software and our knowledge evolve all the time! Questioning things is a good habit.</p>
<p>I've given up trying to look smart. It's being a jerk and lead to nothing. My solutions are meaningless when there is no problem, at least when the effects of the problem have not occurred (yet). So I state the problem, just to see if it's already known and if there's a consensus to do something about it or not. I do not insist. We decide not to do anything for now? That's OK, let's wait for occurrence. I'll offer my help when it will be needed.</p>
<p>The context is important also. First, we are contractors and we bill every change we make to the code. If we find a bug that the clients has not faced yet, it's complicated to inform them and demand money to fix it. It would be like maintaining a ransomware! It's a bad way to do software but the business is OK with that illusion of cost control.</p>
<p>Here is another factor. In our company, people are evaluated for their personal performance, not for the whole team performance. The benefits is that it theoretically push everyone to do its best and be rewarded for it. The problem is that it can easily create unproductive competition between people.</p>
<p>As a new guy in the team I have to figure out who are the leaders. It's useless to get in a fight with them. I propose stuff, and accept that they will be dismissed 80% of the time with a sarcasm. By the way, using sarcasm is lacking factual arguments. It's a good sign!</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-6954651469551143402017-01-04T14:00:00.000+01:002017-01-04T14:00:09.798+01:00Pair programming<p>I appreciate pair programming more and more. It was a controversial practice for me. I used to find uncomfortable to share my computer, my tools, my personal universe with someone. Yet this practice is really powerful. Benefits:</p>
<ul>
<li>It brings more focus. Having a pairing partner, I'm less prone to procrastinate. Even better, I feel like we are less likely to be disturbed by other people.</li>
<li>You catch more typos before running the compiler and / or the tests, hence save sensible amount of time</li>
<li>It's the only efficient way to share practices like TDD, shortcuts and habit that both pairing partners use to be productive</li>
<li>You practice <a href="http://romscodecorner.blogspot.fr/2017/01/code-reviews.html" title="Code review">code reviews</a> on the fly, which is less boring that doing them later (you know later can mean never, don't you?)</li>
</ul>
<p>It's not natural to feel comfortable with pair programming. Here are some advice I can think of :</p>
<ul>
<li>coding dojo is a "safe" way to experience pair programming. However, they attract XP enthusiasts that may already know or use pair programming. You cannot force people to do coding dojo.</li>
<li>use a timer to switch and do breaks. Consider pomodoro technique. I use <a href="http://tomighty.org/" title="Tomighty">tomighty</a>.</li>
<li>if you are more comfortable with pair programming than your pairing partner, pay attention to include her or him in the process. Force yourself to ask questions, ask how does he or she feel. Beware of silence. It's pair programming, not rubber ducking.</li>
</ul>
<p>I know there is a more radical technique in the field today, called <a href="http://mobprogramming.org/" title="Mob programming">mob programming</a>. I'm sure it has benefits in some thorny cases. As I am not a practitioner myself, I don't think it is something you do systematically. And I prefer learning to walk well before running.</p>
<p>What are you telling to your boss to do pair programming? Nothing. Just do it, and measure improvement. You may not be 2 times faster, but you will be faster and improve the overall quality of the code.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-71165680571072998022017-01-02T23:44:00.000+01:002017-01-02T23:44:26.285+01:00Code reviews<p>I have to admit that I made few code reviews in my career. This is bad. Some that I made and some that other dev made on my code focused heavily mainly on the style: comments, ordering of methods, etc. This is <em>worse</em>. You see, lots of bugs cannot be found using unit tests and even performance tests. Examples:</p>
<ul>
<li>hard-coded environnement specific values, like server names.</li>
<li>bad usage of external resources, like opening database connexion in a loop.</li>
<li>shared connections being closed by users, causing invalidation of open cursors (OK, you can catch thiw one with performance testing).</li>
<li>local directory usage on distributed systems. …</li>
</ul>
<p>I faced all of them recently. They appeared in QA when we were lucky. Some of them have been found by the client in production. All of them could have been caught by a correct code review.</p>
<p>Believe me, I will intensify code reviews. Maybe you should too for 2017.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-481132064273790152016-12-12T23:07:00.002+01:002016-12-12T23:07:46.391+01:00Corporate hackathon
<p>Some month ago I participated to a <em>hackathon</em> organized by my company with a colleague. The goal was to deploy a service on a Raspberry Pi 3 that could ingest 1 million messages and provide a synthesis of posted data in JSON format.</p>
<p>It was an occasion to get out of routine work and test new technologies. Here are the solutions we thought of :</p>
<ul>
<li>Python + Redis</li>
<li>Node JS + Redis</li>
<li>Elixir</li>
</ul>
<p>I discarded Python pretty soon, because it was not performant enough. Node looked promising and we found cool stuff to generate the synthesis using Redis. We also tried <a href="https://www.npmjs.com/package/cluster-server" title="Cluster server">cluster server</a>, which gave us tho possibility to run 4 nodes processes. Results where OK on our laptops.</p>
<p>We were waiting for <em>official</em> performance test suite from the organizers 5 using [gatling.io][gat], hence implemented in Scala). It's been buggy for a lot of time and could not rely on it to validate our solution. We were also waiting to be provided with a RPI from the organizers. The final approaching, I decided to buy one.</p>
<p>The tests on the RPI were surprising in the bad way, I did not expect this difference of speed. From 5000 queries per second, we were accepting about 3 times less.</p>
<p>We had another surprises when delivered with the final version of the official test suite : our synthesis was wrong!</p>
<p>I spent 2 nights before the final to deploy a version based on node and postgresql to fix this, as I was sure to be able to compute a synthesis in one SQL query. The request rate dropped bellow 1000 though, which meant we would be able to ingest 1000000 messages in more than 15 minutes. It's pretty long.</p>
<p>Other default, the JSON generated by node was not in expected format regarding numbers. Actually, I put an ugly hack at 3 AM to be able to serialize a Double with 2 digits of precision (you'll see it in <a href="https://framagit.org/rtouze/challenge_iot/tree/master" title="Our code">our code</a>, on node_pg branch).</p>
<p>The <a href="https://github.com/glandais/hackaton-iot" title="Winning implementation">winner</a> of the challenge based his implementation on Java using <a href="http://undertow.io/">Undertow</a> as a web server and <a href="http://www.mapdb.org/">MapDB</a> for storage. The solution ingested the million of messages in less than 3 minutes and 10 million messages in less than 30 minutes when our solution failed miserably after 3 hours of processing! Actually, he has no problem for synthesis generation as the serialisation methods was exactly corresponding with test suite deserialization method.</p>
<p>So we did not as well as we expected, but we learned a lot:</p>
<ul>
<li>Java is not a thing from the past, and concurrency based on multi threading was appropriate here . It's been a great reminder that the relevance of a solution strongly depend on the context.</li>
<li>We should have deployed on RPI sooner to adapt ourselves</li>
<li>Postgresql is a beast hard to tune if you don't have time to. I don't think its appropriate on a RPI. It seems hard to limit IO.</li>
</ul>
<p>By the way, I'm happy with the results since we did not put so much effort in it and we still won a price (a Fitbit blaze that I'm currently trying to sell). Also, I did not have Internet at home at the time, so I did all my tuning offline! It was a great experience and we'll do better next time!</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-6827147462475559152016-11-23T00:08:00.002+01:002016-11-23T00:08:44.482+01:00My talk at Pyconfr 2016
<p>This year I got the chance to speak at the <a href="https://www.pycon.fr/2016/" title="Pyconfr website">Pyconfr</a>! It was motivating for me, as it is a national event and, as you may know, I really like Python.</p>
<p>I was a bit nervous, of course, but as trained myself before the talk, I gained confidence. It's a chance, because I was not so motivated to rehearse. But if I did not, my talk would have been lame. I mean, more that it actually was!</p>
<p>For the record, I submitted 2 talks at the CFP this year. The first one was about <a href="http://hypothesis.works/" title="Hypothesis">Hypothesis</a> and property based testing. I really wished it would be accepted, as it would have motivated me to explore this library.</p>
<p>The one that has been accepted is <em>Help, we to not have any Python project in my company</em>, not so easy to present. You can watch the video <a href="https://www.pycon.fr/2016/videos/au-secours-on-na-pas-de-projet-python-dans-ma-boite.html" title="Mom, it's me on TV!">here</a>!</p>
<p>And, yes, it's in French.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-18797182386230962022016-11-01T00:28:00.002+01:002016-11-01T00:28:57.490+01:00Fed up with properties files<p>Once again, in the source code of our applications, I find lots of
constants defined in a <em>properties</em> file in Java. Why that? In these files, one
can only define <em>strings</em>. Reference to property entries are also strings.
They're not checked at build time. Hence, they cannot be interpreted by an IDE.
You cannot jump easily to your property value. It's a shame, as this is why we
use heavy IDEs! Why are you carrying out such a <em>sabotage</em>??? </p>
<p>In my case, a Java file references constants that are declared in another Java
file where they reference… Properties! That's indirection for the sake of
indirection. </p>
<p>In lots of situations, properties are useful though. You sometimes want
to be able to configure your application without building it again. You want to
be able to setup parameters specific to your running environment. You want to
be able to change some parameters and see them applied without restarting the
application, such as logger configuration.</p>
<p>That represents very few cases though. This is not a good idea to store to
every constant that are used by your application in those files. In most cases,
changing a constant value should be thoroughly tested. It is not something to
be done by everyone, without going into build phase. </p>
<p>So please, stop using properties by default.
Create one only when you cannot avoid it.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-3073425878440417102016-10-25T14:26:00.000+02:002016-10-25T14:26:27.552+02:00Namedtuples in Python<p>Lately, I integrated <a href="https://docs.python.org/3/library/collections.html?highlight=namedtuple#collections.namedtuple" title="Namedtuple in python documentation">namedtuples</a> in my Python programming vocabulary. They allow you to create data structure classes in one line.</p>
<p>Start by importing them from <code>collections</code> module:</p>
<pre><code>from collections import namedtuple</code></pre>
<p>Define a new class as a <code>namedtuple</code>:</p>
<pre><code>Person = namedtuple("Person", ("firstname", "lastname"))</code></pre>
<p>You can know create new instances of Person, as you would do with any other class:</p>
<pre><code>john = Person("John", "Doe")</code></pre>
<p>And if you print it:</p>
<pre><code>In [8]: print(john)
Person(firstname='John', lastname='Doe')</code></pre>
<p>(yes, I use <a href="http://ipython.org/" title="IPython">IPython</a>, don't you?)</p>
<p>Now let's see what namedtuples give you.</p>
<p>Unpacking:</p>
<pre><code>In [9]: f,l = john
In [10]: f
Out[10]: 'John'
In [11]: l
Out[11]: 'Doe'</code></pre>
<p>Field access by name:</p>
<pre><code>In [12]: john.firstname
Out[12]: 'John'
In [13]: john.lastname
Out[13]: 'Doe'</code></pre>
<p>You also have access by index:</p>
<pre><code>In [27]: john[0]
Out[27]: 'John'
In [28]: john[1]
Out[28]: 'Doe'</code></pre>
<p>(OK, I tried some stuff during the redaction of the article)</p>
<p>And that means you can iterate on them, great!</p>
<pre><code>In [31]: for value in john:
....: print(value)
....:
John
Doe</code></pre>
<p>You can retrieve the indexes of defined values (like in tuples):</p>
<pre><code>In [29]: john.index('Doe')
Out[29]: 1</code></pre>
<p>And count the occurrences of the values for free (also like in tuples, which is useless in my example)</p>
<pre><code>In [30]: john.count('John')
Out[30]: 1</code></pre>
<p>There's more. Contrary to classes, equality is defined for you for free:</p>
<pre><code>In [32]: john2 = Person('John', 'Doe')
In [33]: john == john2
Out[33]: True</code></pre>
<p>And last but not least, like standard tuples, namedtuples are <em>immutable</em></p>
<pre><code>In [34]: john.firstname = "Billy"
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-34-a7d9f29302d8> in <module>()
----> 1 john.firstname = "Billy"
AttributeError: can't set attribute</code></pre>
<p>This last one is awesome.</p>
<p>So what is cool with all that? The great strength of tuples is that they are immutable, unlike lists. Thats why, when you have an array of values that is not subject to change, you should consider to create it as a tuple <em>by default</em>. Tuples are memory efficient and give you the insurance that nothing will alter them in your application. Besides, the syntax to create them is a bit shorter:</p>
<pre><code>In [39]: t = 1, 2, 3, 4, 5 # you don't even need the parens!
In [40]: t
Out[40]: (1, 2, 3, 4, 5)</code></pre>
<p>Named tuples extends this ability to any data structure you could create, giving you access to fields by name for readability.</p>
<p>The only drawback is that you cannot define methods or properties on them, as you could do in immutable data structures in other languages. Yet it is still a nice feature of Python.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-55482045449243822932016-08-23T13:30:00.000+02:002016-08-23T13:30:36.408+02:00Playing with Tkinter<p>I committed to produce a GUI for an utility at work. My idea was to use Tkinter module in Python. It was a great pretext to use it for the first time!</p>
<p><a href="https://docs.python.org/3/library/tkinter.html?highlight=tkinter#module-tkinter" title="Tkinter on python.org">Tkinter</a> is a GUI toolkit provided with Python’s standard distribution. It’s great since it avoid the burden of installing an external dependency on target systems. I’m impressed because the development is really simple. I’ve faced far less difficulties that I had with <a href="https://www.wxpython.org/" title="wxPython gorgeous web site">wxWindow</a> in the past (I’m also more experienced though).</p>
<p>Before coding I believed that the toolkit would produce bad looking UI. Actually that’s the case, unless you use themed widgets from <a href="https://docs.python.org/3/library/tkinter.ttk.html" title="ttk on python.org">ttk</a> submodule.</p>
<p>I struggle to find decent documentation as there is too little documentation on Python’s website. I found something relevant <a href="https://infohost.nmt.edu/tcc/help/pubs/tkinter/web/index.html" title="Tkinter documentation">here</a> (doc also available in PDF). Stack Overflow takes care of the rest.</p>
<p>Epilogue: eventually the piece of software will be coded in vb.Net and integrated in a larger app. It’s a shame! By the way I had fun working with Tkinter.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-46847053645066868222016-06-08T17:44:00.000+02:002016-06-08T17:44:10.508+02:00The Software Craftsman<p>I’ve been asking myself if I should buy and read <a href="https://www.amazon.com/Software-Craftsman-Professionalism-Pragmatism-Robert/dp/0134052501/ref=sr_1_1?ie=UTF8&qid=1465400174&sr=8-1&keywords=the+software+craftsman" title="The Software Creaftsman">The Software Crafstman</a> by <a href="https://twitter.com/sandromancuso" title="Sandro Mancuso on Twitter">Sanddro Mancuso</a> given I already read <a href="https://www.amazon.fr/s/?url=search-alias%3Daps&field-keywords=the%20clean%20coder&tag=duckduckgo-ffsb-fr-b-21" title="The Clean Coder">The Clean Coder</a> by <a href="https://twitter.com/unclebobmartin" title="Uncle Bob Martin on Twitter">Robert C. Martin</a>. Both books deal with the same topic: professionalism in software development. Lately, I grabbed a copy from a colleague (thank you <a href="https://twitter.com/JeffLeFoll" title="Jeff Le Foll on Twitter">Jeff</a>) and I read within a weekend (skimming a few pages, I admit).</p>
<p>There’s a lot in common between this book and the Clean Coder of course. What I appreciated though is that being younger than Bob, Sandro writes about experiences close to mines. He talks more about issues on Java projects than bugs in the early stage of computing with punch cards and teletypes.</p>
<p>What did I took from the book? The part about building a learning culture at work. Proposed ideas are great! I regret the overall tone of sarcasm against management though. The book is worth reading for managers too, not to blame them but to educate them.</p>
<p>Do you need to read it if you already read other book on technical agility, like the <a href="https://pragprog.com/book/tpp/the-pragmatic-programmer" title="The Pragmatic Programmer">Pragramatic Programmer</a>, <a href="https://www.amazon.fr/Extreme-Programming-Explained-Embrace-Change/dp/0321278658/ref=sr_1_1?ie=UTF8&qid=1465400369&sr=8-1&keywords=extreme+programming+explained" title="Extreme Programming Explained">Extreme Programming Explained</a> or <a href="https://www.amazon.fr/s/?url=search-alias%3Daps&field-keywords=the%20clean%20coder&tag=duckduckgo-ffsb-fr-b-21" title="The Clean Coder">The Clean Coder</a>? Maybe not. On the other hand, if you just discovered and feel attracted by the idea of software craftsmanship, this book is the perfect companion to start your journey on this path.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-2936142570798340002016-06-03T17:44:00.002+02:002016-06-03T17:44:05.847+02:00TDD By Example<p>I’ve been practicing Test Driven Development for years now. Yet, I haven’t read <a href="https://twitter.com/KentBeck" title="Kent Beck on Twitter">Kent Beck</a>’s book <a href="http://www.amazon.com/Test-Driven-Development-Kent-Beck/dp/0321146530/ref=sr_1_1?ie=UTF8&qid=1464968425&sr=8-1&keywords=test+driven+development+by+example" title="Test Driven Development By Example">TDD by Example</a> until these days. And I have to admit I learned useful things!</p>
<p>First, you all know the 3 steps of TDD: red bar, green bar and remove duplication. But I though that it talked about duplication in the production code. Actually, the most common aspect of duplication when you TDD is between your testing code and your production code. For instance, when you write as the first test for a <em>sum</em> function:</p>
<pre><code>assert sum (1, 2) == 3</code></pre>
<p>Then the code:</p>
<pre><code>def sum(a, b):
return 3</code></pre>
<p>The hardcoded value 3 is a duplication you have to get rid of when having a green bar. It’s nice because you don’t have to write another test to create a triangulation situation (notice that following this logic of triangulation, you could create a test for every different input. That’s stupid).</p>
<p>When I began, I did not understand this idea of the most simple implementation possible that passes the tests. Today, I do it systematically: it allows me to get my first green bar quickly. It’s not that simple because all the code that is called in the test has to be implemented. Now I can give an explanation for this way of doing!</p>
<p>I also learned is that the goal of TDD is not to allow you to write the perfect code during your first sprint (or iteration) though it’s a great too to do so. The test harness allows you to deliver good enough code that will be easy to extend when new features are needed. Most of the time, you do not need an over engineered generic code and it’s OK not to comply at 100% to the <a href="https://en.wikipedia.org/wiki/Open/closed_principle" title="Open Closed Principle">Open Closed Principle</a> at first.</p>
<p>Refactoring is a very pleasant activity that can give a lot of satisfaction: you are in the green, you add abstraction, you implement design patterns to show your skills to your colleagues. But it takes time and you’re not payed to show technical mastery. You’re paid to add value, to help your company to gain more profit!</p>
<p>I had this problem lately. Getting into <a href="https://en.wikipedia.org/wiki/Domain-driven_design" title="Domain Driven Design">Domain Driven Design</a>, I had a tendency to put the patterns in place everywhere. Sometimes, I may have fall into the over engineering trap… Well, that’s learning!</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.comtag:blogger.com,1999:blog-7052813581099080953.post-74071870051062900602016-04-25T12:34:00.000+02:002016-04-25T12:34:34.320+02:00Now I use space as leader<p>Lately, I read an increasing number of articles about using space bar as leader
key to Vim. So I decided to give it a try. </p>
<p>Leader key is a key that can be use in your Vim mapping with the <code><leader></code>
keyword. For instance:</p>
<pre><code>" Typing <leader> then n in normal mode toggles NERDTree
nnoremap <leader>n :NERDTreeToggle<cr>
</code></pre>
<p>So it can quickly become a mapping trigger.</p>
<p>I have to admit I am a recent user of the leader key. By default, it's set to
backslash. You can change it by writing the line bellow in your .vimrc:</p>
<pre><code>let mapleader='<some_char>'
</code></pre>
<p>Backslash is not convenient, even less for French users sticking with French
keyboard layout. So I changed to the comma (as I read it was the way to do). It
makes me begin to use it in my mapping. But I have not using it a lot so far,
as the comma does not get so easily under my finger. </p>
<p>And a couple of days ago, I decided to map it to space. And I got my mind blown!</p>
<pre><code>" doing so is enough
let mapleader=' '
</code></pre>
<p>Obviously, the space bar is the most accessible key of your keyboard, so
mapping can be typed as easily as possible. </p>
<p>The greatest effect is that it makes me think about creating new mappings to
improve my productivity. And I have a lot of fun doing it since then. I shall
update my <a href="https://github.com/rtouze/dotfiles" title="My dotfiles on github.">.vimrc on github</a> soon. I have put some other new stuff in it.</p>
Anonymoushttp://www.blogger.com/profile/02780829884164015350noreply@blogger.com