RSS and Categories
It can't be that difficult can it?......
As part of our implementation for Xtuple, one of the key requirements was to value the stock using FIFO rather than the methods offered by default in Xtuple. After migrating from Netsuite, this became the next major task in our implementation. It took almost a year to develop, initially the first release took around 5 months, but it was only after many rounds of fixes and tweaks that we finally got to a situation where it would correctly calculate the stock valuation and correctly apply this to the General Ledger.
Backups, yes we do backup occasionally, and I've been looking for a better solutions than my historical office<->hosting replication, which although cheap, always made me wonder if I was still making copies. So after our recent office move, and minor server upgrade, I thought better double check on it all.. As usual, the thing had failed due to various reasons, and needed replacing.
So since I've been thinking about a new solution for our clients, I decided to go ahead and try it out for our data. I've been googling affordable cloud based storage for a while and found a company onlinestoragesolutions.com, the pricing is very reasonable US$45 for 2 years currently, with unlimited everything. There are a few review sites that seem to throw some cold water over the offer, but I had one client sign up without any major issues (for another reason), so at that price I though let's give it a go....
One of the key features they advertise is rsync, which since we run all linux machines would be ideal. However after paying, and getting access, I realized it's not quite a simple as pointing your rsync at their server. You need to set up an ssh tunnel to route rsync through.
Can't be that difficult I thought.... Turns out that setting up a password based ssh tunnel, automatically on a cron job, is no small task, there are questions all over stackexchange and various forums, none that I saw managed to find a solution. Most of the suggestions are based around the 'expect' program, a usefull unix tool which can be used to script ssh access. The problem in this case was that setting up the tunnel, then doing the rsync, and then closing the tunnel is not something that a bash, expect or any other method I found could do easily if at all.
So almost at the point of giving up, I started looking around at php's popen (which would not work either), and fell over the pecl expect extension.
Below is the result of a few minutes coding, which does exactly what is needed, and can be run directly from cron. Feel free to escape from overpriced backup solutions..
As the business is always growing, finding time to write even the shortest of blog posts becomes more difficult, but as we are looking for more staff yet again to help solve our clients growth plans, I thought I cover some of our inventive ways we have dealt with growth, and how we differ from most other software development companies.
When I started the original consulting company over 10 years ago, I focused the work on software development which tried to solve complex problems and deliver maintainable systems that could grow over the years. If we needed design work, it would be either outsourced, or done in partnership with similarly focused companies. This means our business has long been based on repeat business from a group of growing companies
Over the last year as we have grown to three people, and still as busy as ever, that focus has become even more clear, as I have written before, our competitive environment consists mostly of design companies with limited interest in producing quality software, rather quickly throwing trash out the door, getting paid, and moving on to the next sucker.. (sometimes we get to clear up that mess...)
Applying software development to our own processes.
Our business is no different in many ways from our clients, we are always attempting to optimize our processes, reduce time spent on meaningless tasks and ensure that everyone is spending their time adding value to our clients businesses. In software development this involves a number of core pieces of infrastructure, Revision control, review systems, task tracking, time billing and accounting. The result and aim of these are two pronged, ensuring that the quality of work being produced is worth the money we are being paid for it, and making sure our clients are aware of their current and outstanding expenses.
While we have the above processes in place, they often are not applied in exactly the same way that the industry has come to expect. This is mainly due to the way that rather than going for off the shelf solutions to our problems, we have tended to develop the tools in-house (a great way to perfect your skills often). While this smacks of NIH (Not invented Here) syndrome, and I can not say that I do not worry about that sometimes, the result is that our processes drive the design of our tools rather that our tools designing our processes.
At present one of the key differences is our approach to the classic developer focus on task driven commits. As this blog has mentioned before, we do not do task driven commits (gitlive). Every single file save is committed into our revision control system. While this may sound a bit strange, It has significant benefits for all involved. After so many years of working I can not tell you how many times that work has been lost due to accidents. Since we are getting paid for this work, any possibility of loosing it would be costing us money (or if we where not so ethical, our clients, which sadly i suspect happens all to often in the industry.).
I have been considering for a while if we should merge this workflow into a more task driven commit methodology, but as what I though would be a short term fix, seems to have re-in-forced this idea that we should not go down that route. This short term fix was the concept of daily commit messages. In a normal task driven process, the commit that fixed the issue would be appended to the ticket (and frequently emailed for review), it could then be reviewed as the issue was signed off.
We did try this on one project, however what resulted was a constant back and forth on multiple issues, and a very time consuming review process. This using the most expensive resource in the company, senior developers. So in switching to daily diff's there was only the need to review the changes for each project. In this way it was also possible to communicate not only serious code fixes, but also advise on better practices. without having to throw that into a tracking system (Although email in some respects is always a tracking system).
The full stack experience
Our new staff over the last year have grown considerably, comming from a classic Hong Kong software development environment, they had ended up focused on a single area of the development process. They would work on projects for weeks and come back with a result, often having to persude, or wait on information from other teams working on a different area of the same project.
I regard this as one of the worst industry practices out here, so over the period they have been working at Roo J Solutions, they have been thrown in the deep end with every part of the standard development stack, front end, back end and database. Tasks are frequently day or even hour based, however working as a team now, they can pass the learning of the new tools around. Always having my knowledge as a good backup. To the suprise of someone who had worked on that team with them before, they not only did not sink, but swam very well being given new challenges.
So if this sounds like a interesting place to work and you or someone you know would like to be attacking our challenging new projects, please pass on the post or email me at firstname.lastname@example.org, with the usual information (If I have to tell you.....)
If there is one reason I would put down to why I still enjoy coding after all these years, it is the pleasure finding a really challenging problem and being able to solve it quickly and efficiently.
This week saw one of those problems, strange, obtuse, and to most non crazy people would seem to be an odd thing to get excited about. It all started with going through the requirements list on one of our projects.
As I've mentioned before, we are building quite heavily ontop of Xtuple, a Web based GUI using the RooJS Toolkit, most of that part has been just process orientated - add this feature, and it just works. But some of the implementation process has been a challenge.
The company we are working with has 2 main offices (and more comming on line soon), and we have successfully migrated them off of netsuite, which in reality made an absolute mess of their accounts. When migrating to Xtuple (which in essence is a postgresql database, with a seemingly endless number of table triggers, stored procedures and views), we concluded quite early on that setting up a seperate instance for each office would be the best way to go.
Core issues like difference currencies in each office, so the chart of accounts can be reported each year in the local currency to the tax department and auditors, along with the different end-of-year tax reporting made this a simple decission. The otehr issues was that the basic installation of Xtuple does not really support Multiple companies in the Desktop UI (you have to pay for that feature).
So as we are now live, and just about finished with the rollout issues, we are now on to the more critical things for long term operations - Management reports (and a nice dashboard).
The issue that raised it's head was that the company needs to see consolidated Balance sheet and Income statements. It did not take to much time to call the internal methods in Xtuple 'financialreport()' and in PHP convert this to JSON so that a master script can call the report in both systems, and merge them together.
The snag came later when we realized that the resulting report was a little off.. Due to the magic of differeing accounting periods.
This week saw the release of the latest site developed by Roo J Solutions, www.prettybooked.com a Hong Kong based beauty booking service. While the site has gone through the usual last minute bug-fixes before going live, it all came together in the end.
Developing a feature rich web site.
While we where busy finishing off this site, We have had a few other projects and discussions going on, and what has been interesting lately has been the realization that far too many companies and startups are having real trouble getting web sites developed properly.
Often they are cutting costs, and contracting under-qualified or unsuitable individuals for the process. Resulting in disaster hitting quite early and the project never being complete as their 'developers' have no idea how to solve the technical issues involved.
But the saddest and worst scenario's I've seen, has been the frequent employing of design companies or 'interactive media' companies as the lead contractors on web based development. This can be the most expensive recipe for disaster. In one case the cost of the development was bloated to over 10 times the realistic cost of the project, and what was delivered was just a prettified open source project. When the client started asking for features that the original open source project provided, they where quoted quite unreasonable amounts. The sad fact is that the relationship between the supplier and customer was damaged beyond repair. Without a deep understanding of how software projects are developed, these companies failed to manage their customer's expectations.
Fool me once, shame on you; fool me twice, shame on me.
In another case a design company was asked to provide a custom web site, which from a design perspective looked quite nice, however as part of our installation on the server, we got to see the kind of code that was being delivered. If this software was a car, the wheels would fall of when driving out the garage, It was the worst and most dangerous code I've seen in nearly 10 years of development.
Read on for more about prettybooked, and how to get projects done properly...
One of the more interesting projects I have ongoing is a migration and deployment of an ERP system. For a bit of background, our customer was persuaded about 3-4 years ago that moving their accounting and Stock management into Netsuite would be a good idea.
For those who have not heard of Netsuite, it's a Cloud based ERP system, that from my understanding runs on an oracle backend and is totally web based. All the users log into the website, enter the various stock activity ( Purchase Orders, Item Receipt, Bills, Sales orders, Invoices, Item Fulfilments and Various Credit Memos), along with the standard accounting features like Journal entries. etc.... The system keeps track of your stock and should give you a value of stock on hand, along with doing your audit-able reports.
Well, I'm guessing anyone who has worked with ERP systems, probably knows that the devil is in the details for this. The company I have been working with have been using this for over 2 years now, and are beginning to see the ugly side of cloud based system (which is why we are trying to migrate off of it.), Let's have a look through the issues.
Having written a template engine for PHP, I can pretty much say that not using a template engine (which automatically escapes output) is essential for outputing HTML. It makes everything more maintainable, and reduces the risks that as a error prone human being, you are likely to open your application up to exploits.
Read on for my opinion / review of Anjular and the introduction to the updated Roo.XTemplate
With a nice long Easter holiday, I finally got a chance to hack on some of those todo items that I'd been putting off for quite a while. One of the core mini tasks was to enable mtrack information inside of my main email/accounting do everything platform.
The Pman codebase, as I've mentioned before forms the core of pretty much all applications I work on now, mostly intranet or extranet focused web applications, that work like desktop applications. All of these applications are built up of various componenents, for example the Accouting module has Components like general ledger, managing invoices, timesheet and tracking editing and a pricing management component.
In the case of the Accounting module, there is a default top level module called 'Pman', which the Accounting 'Tab' is added to, then all the sub components are added to that. This whole design was done a couple of years ago, and has been working very well, it's reliable and adding extra componenets, is just a mater creating a file then saying what the parent is and in what sequence it will appear.
Read on to find out how this all works...
Since we have been very busy already this year, I have now almost completed the process of migrating from a Sole Proprietor into a Limited Company. Roo J Solutions Limited is now a registered Hong Kong Company. We are now looking for full or part-time staff (based in Hong Kong).
Please read the full post for details.
One of the great things about the internet is the availability of cheap or free services online, so many clients are using gmail, dropbox, github etc. for their business operations. But all to often they forget that these services are often playing the oldest game in the technology industry. "Vendor Lock-in".
While the ones I mentioned are not to bad, you can cheaply and easily rescue or backup your data to another location, or move to an alternative provider. Not all of them are like that.
We are in the middle of a migration project from Netsuite (It's a SAS Oracle based ERP system) to Xtuple, which is a open source ERP system, based around postgresql. This is a slow and painfull migration, as there is no standard for ERP data, and exporting is slow and clumsy over SOAP. Anyway, as a plesant distraction from this large migration, the same client also wanted us to look at migrating from backpack, a 37 signals product.
Backpack, unlike all the SAS systems I mentioned has deliberately made it hard, or practically impossible to migrate from their services. The primary offering of backpack is a online file storage service that you can permit clients or suppliers the ability to do share files and folders. It is only web based (unlike dropbox or box.net), and there is no desktop client that you can use to access the files other than the web interface.
So how to rescue the data... Read on for the trick..
- FIFO on xtuple
- PHP just does some things better. cloud backups, pecl-expect
- Roo J Solutions Growing into 2013, always recruiting, and developing differently
- Interesting Problems.
- Prettybooked.com turning a startup into a reality.
- Migrating off Netsuite - The hidden cost of Clouds..
- Roo.XComponent introduction