Labels: c++ (6) scripting (4) the perfect project (3) coding (1) firefox (1) idea (1)

Friday, February 15, 2008

Open source is good for your health

Linux helps to code for Windows too

What exactly Variant.ChangeType( VT_NULL ) does? Will it free any memory owned by variant?

Common sense suggests "yes", but looking at all these low level struct-union-macros games I cannot avoid being paranoid sometimes. Unfortunately, MSDN help does not have such level of details and does not have any notions specifically about VT_NULL usage here.

I am glad I've found Wine sources and could check their logic. Assuming their VARIANT_Coerce mimics Windows API logic I can be sure now that memory is freed. Hooray! I do not have to debug _variant_t!

Friday, September 28, 2007

Idea: background building of Visual Studio C++ projects

Sometimes I come up with crazy ideas about programs or scripts that could do magic.

Since I don't have time or knowledge to implement them all, I decided at least describe them here. Hopefully somebody else finds them useful and decides to make them a reality. In this case, I will be the first to help. When you think of something long enough, it gets more and more details, single blog post cannot describe them all but just some key points only.

CPP background compiler

Ok, today's idea is about compiling source files. I often wish that Visual Studio compiled projects on a background, while you editing them. Of course, the changed files would need to be recompiled. Updating headers would require many cpp files to be recompiled... So what? Those files will take their time anyway, when you press "Build". But if you have most of the files compiled on a background, building the project from IDE would be much faster, since only the recently changed files would need to be compiled and only linkage would left.

Simple and stupid approach: build it all

Just pick up the solution or project and run a batch file which would constantly call devenv with /build switch to attempt to build the project. Make that process idle priority, so it would not affect other programs much.

Visual Studio doesn't like much if you're saving source files during compilation. If that is a cpp file which was saved, then during next build request, it should figure out that obj file has creation time less than cpp and recompile it. Not a big deal, but this shall work only if my following assumptions are true:

  1. Compiler loads entire cpp file into memory before compiling it (including preprocessing)
  2. Obj file creation time equals to the time when file was started to be written on disk, not when it was closed
  3. The time between cpp was loaded into compiler's memory and the time obj file was opened on disk is negligibly small (or even better — obj file is opened before cpp file was read from the disk.

Of course, modifying header may cause more recompilations, since Visual Studio must figure out what cpp files depend on it and recompile them. I don't see much problems, as soon as the assumptions above are ok.

Well, almost no problems. If you update stdafx.h this means you should recompile stdafx.cpp. And if you're recompiling stdafx.cpp this means you should recompile all cpps in the project. I am not sure Visual Studio /build is smart enough to figure that out, so this is better to be tested.

One small thing:

Synchronization with Visual Studio IDE

Having another instance of compiler writing some obj file will prevent Visual Studio to compile that file from IDE. This means we should synchronize IDE with our background script. Practically this means we should stop or kill that background IDE before building project from IDE.

I see this could be done by creating a macro, which would call pskill.exe (written by Mark Russinovich from Sysinternals) before actually starting a build from IDE. To avoid running background compilation script manually after IDE build finishes, we could run it from project post-build event. However this will only work if IDE compilation was successful. Otherwise — alas, I don't see how to call a macro after build finishes with error. Maybe creating IDE add-on would help?

Ok, that seems not too efficient to build an entire project on background, since it will do scanning all of the project files and sometimes that can be a lot. The synchronization problems with IDE make it worse too.

How about we compile only cpps?

We could run compiler itself (cl.exe) on all cpp files of the project, just like IDE does. It would require to pick up project options from vcproj file, also we would need to clean the objs after compiler options were updated in vcproj (and don't forget about IDE environment variables). That seems like more difficult to do, but still nothing too special.

This way IDE would do the linkage and all post-build stuff and our background script would only compile the cpps as they get updated.

Synchronization would be less required, since the background script would do nothing after all cpps are compiled. So every time you start the build, it could just wait until the background script finishes compiling all the recently updated cpps and then start IDE build (presumably only linkage would left at this point). As a signal that he is done, the background script could create an empty file with predefined name in project directory, which could be easily checked by macro for existence. Also, this macro would need to call "SaveAllFiles" manually so background script would not wake up after IDE saves the modified source files before starting a build.

The only thing which makes complicates this approach more is that background script should watch header files too and recompile all affected cpps (having stdafx in mind too), i.e. reproduce more of the IDE logic. Having many nested include files makes this task not so trivial. However still doable.

Sounds familiar?

If we can do pretty much what IDE does with the builds, maybe we could leave it only editing of files and projects and avoid synchronization problems just by relying on background compilation scripts?

We could remap standard shortcuts so they pass commands to our "engine" and even redirect it's output to IDE build windows probably. If that cannot be done with macros, maybe add-on could work instead.

Another thing I often desperately need in Visual Studio is queueing the projects compilation. Usually this is required when you have big solution and few projects you want to be built. You start "Build this project only" on one project and then wait patiently until it finishes, then you run nother project the same way... Of course you can create build configuration and group such projects there. But sometimes it is just "this and that" and not worth creating a configuration for a single case.

So, at this point having a command "add to the building queue" would be perfect, but there is no such thing in IDE (at least in Visual Studio 7.1 aka 2003). But if we had a script-engine capable of building projects, adding some queueing to it would be pretty simple and convenient.

This is pretty close actually to having own IDE for C++ projects, but that is another big idea. =)

Let me know if you implement some of these approaches. Of course, if I ever do this, I will write about it here. Maybe having a plan and design in form of a blog post will make this come true sooner — we'll see…

Friday, July 27, 2007

Requirements for TPP

This is the first part of requirements for the perfect project. It turns out they are "level 0", i.e. the most essential ones. More requirements shall be published later and specify user-generated content features, internationalization and maybe some insights about further functions evolution.

Glossary

TPP, the perfect project = code name for this program I want to develop. Later this can be changed by other name.

Program, Software = in most cases any 3rd party software which shall be shipped and managed with the help of TPP. This doesn't include any system libraries, video codecs or anything like that. Note: This may include some scripts which actually only configure existing software, update some settings or change behavior of the programs already installed. Note: The program means separate functionality unit which can be independently installed or uninstalled. This shall not be a module extending the functionality of the existing program, because it will introduce dependencies between modules if main program is being uninstalled and dependant module wasn't. User shall not know or decide about these dependencies during install or uninstall (however she can select packages of software to install as single program using TPP). Alternatively, for some add-ons, we could have them installable to "main" host program, but still consider uninstall as atomic operation for host program with all its add-ons altogether. More examples are needed to think this all through.

Installation, Program installation = Installation and configuration any program, the process which includes downloading and copying all necessary program files, registering the software within operating system and existing software environment and configuring all program settings which are required for user to work with the program.

Rebooting, operation system rebooting = The actions which are needed to be performed during program installation which assume giving all control from TPP back to operating system. Usually this means shutting down the operating system and starting it again automatically and TPP taking control again after some moment. During rebooting, the user may be asked to log in to operating system before TPP activates. User can also postpone the installation for any time simply by shutting down the computer during rebooting, or by logging in the operating system using account other than he used to work with TPP.

Safe rebooting, rebooting safely = shall be defined later and depend whether the user have opened programs which cannot close automatically without losing the user data.

TPP Requirements (F = functional, NF = non-functional)

Browsing programs

  1. (F) TPP shall allow the user to browse available programs.
    1. (NF) User shall be required to have an internet connection to browse and install programs with help of TPP.
    2. (F) TPP shall allow the user to select and clarify his selection of category of the program he is looking for from the pre-defined catalog.
    3. (F) The catalog of programs should allow the user to find the program:
      1. (F) Using keywords describing desired functionality. For example: graphics editor, photo editor, antivirus software etc.
      2. (F) Using keywords describing the program's description or name as provided by author.
      3. (F) Using keywords describing the program as given by other users.
      4. (F) Using keywords describing desired task that should be accomplished with the help of the program. For example: write CD, copy DVD, print book, draw picture, record movie, copy movie, protect from viruses etc.

Installing programs

  1. (F) TPP shall allow the user to install the selected program automatically.
    1. (F) User shall not be asked for any parameters during program installation.
      1. (F) User shall not be asked for the permission to download the program from internet or any other resource.
      2. (F) User shall not be asked the path where to install the program.
      3. (F) User shall not be asked to remove older version of the software being installed.
      4. (F) User shall not be asked to install other prerequisite libraries or programs required by software being installed.
      5. (F) User shall not be asked to accept any license agreements.
      6. (F) User shall not be asked to customize functionality package for the software being installed.
      7. (F) User shall not be asked the language of the program interface.
      8. (F) If the program user selected to install already installed, the program shall be re-installed as it was a newer version of the software, without any confirmations (see req. 2.1.2).
    2. (F) If the program installation required rebooting the computer, the user shall be notified about that before installation starts.
      1. (F) If rebooting can be done safely, the user shall not be asked for any permission to do it.
      2. (F) If rebooting cannot be done safely, the user shall be asked the permission for it before installation starts.
    3. (F) At any point during installation except the time when operating system is being rebooted, the user shall be able to cancel the installation.
      1. (F) When installation is cancelled, all the changes made during the installation shall be rolled back to the state as they were before installation started.
        1. (F) User shall not be asked to confirm the rollback.
        2. (F) If rollback requires rebooting again and it can be done safely, it should be done without asking the user for permission. Otherwise the user shall be asked for the permission to reboot.

Browsing installed programs

  1. (F) TPP shall allow the user to browse installed programs on his computer using the same functionality as described in requirements 1.
    1. (NF) User shall not be required to have internet connection to browse and view existing installed software.

Uninstalling programs

  1. (F) TPP shall allow the user to uninstall any program installed using TPP automatically.
    1. (F) Uninstalling the program shall not leave any files or folders created as the part of its work unless those files are documents created by user and manually saved by him at location where the program was installed. For example: program uninstall shall not leave custom setting files, shared libraries, shared plug-ins, game save files etc.
    2. (F) User shall not be asked for any parameters during program uninstall.
      1. (F) User shall not be asked for confirmation to uninstall the program.
      2. (F) User shall not be asked for confirmation to remove some files or libraries, shared or not shared by other applications.
      3. (F) User shall not be asked to rollback to older version of the software after uninstalling the selected one, if the selected one was installed over older version.
      4. (F) User shall not be asked for any feedbacks to the creators of the program or anybody else.
      5. (F) User shall not be asked to remove only some components of software instead of uninstall.
    3. (F) If program uninstall requires rebooting the computer, the user shall be notified about that before uninstall starts.
      1. (F) If rebooting can be done safely, the user shall not be asked for any permission to do it.
      2. (F) If rebooting cannot be done safely, the user shall be asked the permission for it before uninstall starts.
    4. (F) At any point during uninstall except the time when operating system is being rebooted, the user shall be able to cancel uninstall.
      1. (F) When uninstall is cancelled, all the changes made during uninstall shall be rolled back to the state as they were before uninstall started.
        1. (F) User shall not be asked to confirm the rollback.
        2. (F) If rollback requires rebooting again and it can be done safely, it should be done without asking the user for permission. Otherwise the user shall be asked for the permission to reboot.

Sunday, April 15, 2007

The perfect project: Problem formulation

Ok, we are gonna do some software project and do it as best as we can (Let's use "we", cause I don't like using "I" all the times. Anyway if you are reading, then you are participating already). Best practices, processes, technologies — all of that is welcomed until it is useful. And perfect. =) So, what are we gonna do? Let it be something simple enough to not stuck in technical details, but complex enough to have value and serve as a playground for demonstrating the processes and stuff. Let it be... Fibonacci numbers calculator. No, even better — factorial number calculator! Or, maybe a converter of temperature between Celcius and Fahrenheit? This is the first stop. That is the problem of 99% of the books about programming we have. People read them to learn programming, to learn how to create things. And they start with the wrong examples.

Technology Review: Why is most software so bad?

[…]

Technology Review: How can we fix the mess we are in?

Bjarne Stroustrup: In theory, the answer is simple: educate our software developers better, use more-appropriate design methods, and design for flexibility and for the long haul. Reward correct, solid, and safe systems. Punish sloppiness.

When a student starts to learn programming, what book it is pointed first? Yes, a programming language book. Like C++ Programming Language. And what happens after that? On conferences like SD West 2007, Bjarne Stroustrup and Herb Sutter — the gurus, the leaders of software development world — spend hours explaining software developers (adults now) the things they do wrong. Like why they should avoid using C-arrays and favor std::vector. That's ridiculous! And just in the middle of that (around $175 per hour BTW), each of them uses fopen/fclose on their slides to demonstrate the value and proper usage of constructors/destructors and smart pointers: Bjarne: And Herb: While we have such stupid (std::ifstream hopefully will be revealed on SD West 2008) examples, we are gonna have developers (students yesterday) using these examples in real world projects. Because examples are supposed to be followed and are followed, no matter how "sloppy" they are.

The perfect project: so what are we gonna do?

WHO CARES about calculating the Fibonacci numbers? Who needs those silly, artificial and useless examples? If you ever need those, go and find them. Google them, lookup in Wikipedia or whatever. But real programs should be useful and provide value for people. To make our project perfect, we are gonna find something useful and not done yet. Something you cannot just buy instead of constructing — thanks to Frederick Brooks for the idea! Let's spend some time thinking about what that could be.

The perfect project: Start

Every software project is a part of developers' lives. We are spending our time through the usual processes from task formulation, project inception, requirements, design and coding to the testing and ongoing development of a new baby of ours. It can be a whole new module of the system, or just a modification of it, but it goes all the same. That's our lives. That's our legacy. Should we care about how it goes? Sometimes I feel desperate because I think I am doing something useless, or with low value, or ineffectively. Sometimes almost everything around seems so imperfect and clumsy that I wonder how we come up with this ugly world of computers, programs and designs. There's always options. Either waste this hour, and that hour for this stupid task, or do something with it — rethink, automate or (even better) eliminate at all. I believe there should be a way to improve the world of software development and the world as the whole. There's should be a way to provide identification and accumulation of human experience and avoiding same errors repeating again. I'm gonna try to form and store my thoughts here. Haven't come up with the better title of it as "The perfect project". That's right, no capital letters, let's start with the small ones.

Monday, April 9, 2007

Any bool operators? (real example)

if ( !spResults.empty() && !spResults->empty() )...

Readable exceptions fun

#include <stdexcept>
#include <iostream>

class Class
{
public:
   struct Exception
   {
      struct Any : public std::exception {};
      struct CannotConstruct : public Any {};
   };

   Class() { throw Exception::CannotConstruct(); }
};

void iKnowAboutClassAndItsConstruction (void)
{
   try
   {
      Class();
   }
   catch (Class::Exception::CannotConstruct&)
   {
      std::cout << "got Class::Exception::CannotConstruct.\n";
      // Do something... or:
      throw;
   }
}

void iKnowAboutClasses (void)
{
   try
   {
      // do something useful and specific with Class(es):
      iKnowAboutClassAndItsConstruction();
      // ...
      // ...
   }
   catch (Class::Exception::Any&)
   {
      std::cout << "got some Class::Exception.\n";  
      // Do something... or:
      throw;
   }
}

void main (void)
{
   try
   {
      // let's do something with Class(es):
      iKnowAboutClasses();
   }
   // main -- don't care about any specific exceptions but standard:
   catch (std::exception&) { std::cout << "got std::exception.\n";    }
   catch (...)             { std::cout << "got unknown exception.\n"; }
}